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Executive Summary 


The 4-year federal Safe and Supportive Schools (S3) program supports targeted interventions 
to improve and measure conditions for learning at the high school programmatic level. In the S3 
program, “conditions for learning” refers to school climate, defined as “the quality and character of 
school life” reflecting “norms, goals, values, interpersonal relationships, teaching and learning prac- 
tices, and organizational structures” within the school community. 1 

The National Implementation Research Network (NIRN) defines implementation as “a speci- 
fied set of activities designed to put into practice an activity or program of known dimensions.” 2 
NIRN also describes implementation as a developmental process that occurs in discernible stages: 
exploration, installation, initial implementation, and full implementation. In addition to progress- 
ing through these stages, well implemented interventions tend to share common drivers, which fall 
into three categories: competency, organization, and leadership . 3 

Building on this conceptual basis, the WV Model for Positive School Climate (WVMPSC) 
serves as the framework for West Virginia’s S3 project activities. The model provides a systematic, 
data-driven planning process that enables schools to identify and implement effective interventions 
to improve conditions for learning and ultimately to improve student outcomes. It involves strategic 
steps in these broad areas: (a) organizing, (b) assessment, (c) building support, (d) planning and im- 
plementation, and (e) monitoring and evaluation. For each strategic step, a set of core activities help 
schools carry out their school climate improvement process. Appendix A displays how the develop- 
mental stages, implementation drivers, and strategic steps function together (page 29). The WVDE 
Office of Healthy Schools enlisted and trained a cadre of school climate specialists (SCSs) to guide 
teams and leaders from schools selected for the program (S3 schools) through the components of the 
WVMPSC. Each SCS serves from one to eight S3 schools. 

The S3 project has an evaluation component spanning the full 4 years of the project. For this 
report, the WVDE Office of Research focused the first two years of the project— the 2010-11 and 
2011-12 school years— to assess two evaluation questions: (EQi) To what extent do participating 
schools implement the program with fidelity relative to the WVMPSC, and (EQ2) To what extent do 
program initiatives improve school climate and culture? 

Methods 

To assess implementation fidelity, we developed 4-point rubrics for each core activity with 
rating scales that ranged from missing to strong fidelity, but also included additional response op- 
tions of too early to tell and don’t know. The implementation fidelity assessments are completed to- 
ward the end of each academic year by two groups: the cadre of SCSs and school-based leadership 


1 National School Climate Center (n.d.). School climate. New York: Author. Retrieved from www.schoolclimate.org/ 

climate/ 


2 National Implementation Research Network (n.d.a). Learn implementation (online publication). Chapel Hill: University 
of North Carolina, FPG Child Development Institute. Retrieved from http://nirn.fpg.unc.edu/learn- 
implementation/implementation-defined . 

3 Fixsen, D. L., & Blase, K. A. (2008). Drivers framework. Chapel Hill, NC: The National Implementation Research Net- 
work, Frank Porter graham Child Development Institute, University of North Carolina.. Retrieved from 
http://nirn.fpg.unc.edu/learn-implementation/implementation-drivers 
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teams in each S3 intervention school (S3 teams). SCSs submitted one assessment response for each 
school they served; members of the S3 teams collaborated to complete one consensus rubric for their 
schools. 

To assess school climate improvement, two approaches were taken. First, we compared over- 
all school climate index scores for S3 intervention schools between the 2010-11 and the 2011-12 
school years to assess change over time. Second, we added a question in the spring 2012 administra- 
tion of the WV School Climate Surveys for students and staff. In the question, a series of items corre- 
sponding to 20 indicators from the school climate index were listed, and respondents were asked to 
indicate whether the items have changed compared to the previous year. 

Ancillary analyses looked at how much support was provided by school districts and school 
principals, and how well schools crafted their behavioral norms and expectations, a key component 
of the WVMPSC. 


Results 


Fidelity of implementation 

Overall the 15 (of 22) intervention schools for which fidelity assessment data were available 
in both 2011 and 2012 appeared to have done well in implementing core activities that align with the 
exploration and installation stages of implementation. Moreover, competency and organization im- 
plementation drivers tended to be largely in play during the first 2 years of the project as school- 
based teams were established, SCSs provided training and developed coaching relationships with 
school teams, and data systems to support decisions were employed through core activities (i.e., en- 
vironmental scans and SWOT analyses). The use of other relevant data sources (e.g., school climate 
survey data) was initiated and sustained over the 2 years to support school improvement decisions 
and planning. Across most core activities— aligned with all stages of implementation— improvements 
were made, moving from being altogether missing or implemented with weak fidelity in 2011, to be- 
ing implemented at weak to moderate fidelity in 2012. 

Both school-based S3 teams and SCSs indicated marked improvements relative to the strate- 
gic steps of the WVMPSC over the 2 years. SCSs tended to be more guarded in their assessments, 
however. For some core activities they indicated schools’ implementation fidelity to be at lower levels 
than the school S3 teams rated themselves; this was likely due to the broader frame of reference SCSs 
have across schools. Statistically significant differences were found on seven WVMPSC core activities 
in 2012, compared to only three in 2011. The two groups differed— with S3 teams providing higher 
ratings— on two core activities in both years: (a) informing parents and community partners about 
the S3 initiative and securing their commitment, and (b) building understanding of S3 behavioral 
norms among school staff. The first of these is a challenge throughout the entire school system, 
whether involved in S3 or not. The latter is of critical concern because successful school climate im- 
provement relies heavily on the awareness and understanding of expectations for appropriate behav- 
iors, and execution of practices that reinforce those expectations in the day-to-day operations of the 
school. 

A related activity on which the two groups differed in 2012 was defining school-wide proce- 
dural expectations for addressing student behavior. That SCSs assessed these activities at lower lev- 
els of fidelity than the S3 teams points to the need for competency-building opportunities for school 
team members and their colleagues, and for SCSs to coach them along. 

Other core activities that SCSs rated at statistically lower levels than did school S3 teams in- 
volved the use of decision-support data systems. These activities included using assessment results 
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to (a) identify factors contributing to school climate problems, (b) set priorities or plan activities, and 
(c) select appropriate interventions. Differences also were found for schools’ efforts to review or ana- 
lyze data on an ongoing basis to make project adjustments. It is not enough to simply identify prob- 
lematic school climate issues. Achieving the best possible climate improvement also depends heavily 
on selecting and implementing interventions based on thorough assessments of factors contributing 
to those problem areas. 

Leadership implementation drivers are not as explicitly described in the strategic steps of the 
WVMPSC as are competency and organization drivers. Two areas where SCSs provided limited data 
were the role and involvement of district S3 contacts and of school principals. SCSs reported that for 
most intervention schools, the district S3 contacts had at least some involvement in the S3 project at 
the school level, and for eight schools a fairly high level of involvement was noted. For eight other 
schools, however, district coordinators were reported to have had little involvement beyond serving 
as the contact for the grant. There could be valid reasons for varying levels of involvement. In some 
cases, district contacts may be confident that schools have the capacity to carry out the grant with 
little assistance, while for other schools they may believe the opposite to be true, and thus gauge their 
involvement accordingly. It is not clear from the limited findings in this study if district contact in- 
volvement has a positive or negative effect on schools’ success. 

On the other hand, research has shown that in modifying a school's climate and culture, 
school administrators can play a substantial role in clarifying and consistently modeling the beliefs 
and values embodied in any proposed changes in the school environment. 4 They can create the con- 
ditions in which students and staff internalize new practices, similar to the behavior expectations 
established through the WVMSPC. Although it is not clear the specific roles of school principals 
among S3 intervention schools, SCSs reported that in a large majority of schools, principals had been 
very or moderately involved in the S3 project, which bodes well for the project as a whole. There were 
a small number of schools where principals were reported to be largely uninvolved, a possible sub- 
stantial barrier. 

Although no specific guidance is provided in the WVMPSC on the appropriate number of be- 
havioral norms or expectations for a school, a similar framework suggests that three to five clearly 
stated norms/expectations may be optimal. 5 They should be (a) stated clearly; (b) 1-3 words in length 
each; (c) positively stated to promote and support appropriate behavior; (d) comprehensive (apply to 
all students and staff, and all settings); and mutually exclusive. When asked to report on the behavior 
norms or expectations established through the S3 project, 21 of the 22 intervention schools respond- 
ed. The number of behavior expectations listed ranged from two to 10 per school. While 13 listed five 
or fewer as recommended, eight schools listed from six to 10. Recommendations are offered for re- 
fining the behavior expectations established by schools. 

School Climate 

As a group, S3 intervention schools showed significant improvement with medium to large 
effect sizes in school climate as measured by the WV School Climate Index. Differences also were 
found for eight of the 20 indicators the Index comprises. It will not be possible to determine if the 


4 Beach, R. H., & Lindahl, R. A. (2007). The role of planning in the school improvement process. Educational Planning, 
16(2), 19-43. 

5 Algozzine, B., Horner, R. H., Sugai, G., Barrett, S., Dickey, S. R., Eber, L., Kincaid, D., Lewis, T. & Tobin, T. (2010). 
Evaluation blueprint for school-wide positive behavior support. Eugene, OR: National Technical Assistance Center on Positive 
Behavior Interventions and Support. Retrieved from http: / / www.pbis.org/ evaluation/ evaluation blueprint. aspx 
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improvements in the Index observed in this year’s study are genuine until data are collected for the 
full 4 years (including a comparison group of nonintervention schools). 

Based on survey data, there appears to be a fairly wide gulf between students and staff in 
their perceptions of school climate improvements. Students were much more likely to report that 
conditions stayed about the same compared to the previous year, whereas staff were much more like- 
ly to report conditions had gotten better. Discrepancies between student and staff perceptions of 
school conditions are not unheard of. One study found that although school staff were more likely 
than students to report having witnessed bullying in the school environment, they also were more 
likely than students to report feeling safe and a sense of belonging— both important constructs of 
school climate. 6 Another research study reported no association between student and staff ratings of 
overall school climate. Further, teachers’ ratings of school climate may be more sensitive to class- 
room-level factors, such as disruptive behaviors, while students’ perceptions may be more sensitive 
to school-level factors, such as student mobility, student-faculty ratio, and a change in principal. 7 
With staff and students so far apart in their ratings of school climate improvement in the current 
study, there is a question about what factors may be driving the perceptions of both groups. Moreo- 
ver, once identified, could these factors be targeted to further enhance schools’ efforts in school cli- 
mate improvement? 

Recommendations 

1. Based on the list of behavior expectations provided by intervention schools, schools should direct 
attention to establishing or refining behavior expectations with the assistance of the SCSs. Expec- 
tations should be five or fewer in number; 1-3 words in length; positively stated; comprehensive 
(apply to all students and staff, and all settings); and mutually exclusive. The findings from the fi- 
delity assessment also indicate that schools should place additional emphasis on efforts to build 
understanding of behavior expectation among school staff. 

2. Schools also should consider expanding their approaches for communicating and teaching behav- 
iors that align with their expectations. By articulating specifically what appropriate behaviors look 
like relative to the established behavior expectations, developing specific lesson plans to guide the 
process of teaching appropriate behaviors, and modeling appropriate behaviors throughout the 
school environment, they will enhance the degree to which expectations are integrated into day- 
to-day school experience. 

3. Essential to obtaining the best possible climate improvements is selecting and implementing in- 
terventions based on thorough assessments of factors contributing to problematic school climate 
issues. That SCSs assessed schools at lower levels of fidelity in this area than did school teams 
points to a need to redouble schools’ efforts to explore these contributing factors and adjust their 
interventions accordingly. 

4. The gap between students’ and staffs’ perceptions of school climate improvement suggests an op- 
portunity to cultivate a deeper understanding of the school environment from the perspectives of 
these two groups. Identifying and leveraging factors driving the perceptions of both groups could 
provide direction for further efforts to improve school climate. 


6 Waasdorp, T., Pas, E. T., O'Brennan, L. M., & Bradshaw, C. P. (2011). A multilevel perspective on the climate of bullying: 
Discrepancies among students, school staff, and parents. Journal of School Violence, 10(2), 115-132. 

7 Mitchell, M. M., Bradshaw, C. P., & Leaf, P. J. (2010). Student and teacher perceptions of school climate: A multilevel 
exploration of patterns of discrepancy. Journal of School Health, 80, 271-279. 
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Introduction 


The West Virginia Department of Education (WVDE) Office of Healthy Schools was 
among ll state educational agencies that in 2010 were awarded 4-year federal Safe and Support- 
ive Schools (S3) grants. The intent of the S3 program was to support the measurement of— and 
targeted programmatic interventions to improve— conditions for learning at the high school 
programmatic level. In the context of S3, conditions for learning refers to school climate, de- 
fined by the National School Climate Center (NSCC) as “the quality and character of school life” 
reflecting “norms, goals, values, interpersonal relationships, teaching and learning practices, 
and organizational structures” within the school community (NSCC, n.d.). 

The WV Model for Positive School Climate (WVMPSC) serves as West Virginia’s frame- 
work for carrying out the S3 project. An underlying assumption of the model is that it provides a 
systematic, data-driven planning process by which schools may identify and implement effective 
interventions to improve conditions for learning and, ultimately, to improve student outcomes. 
The WVMPSC involves a set of strategic steps, including (a) organizing, (b) assessment, (c) 
building support, (d) planning and implementation, and (e) monitoring and evaluation (Figure 
1). Within each of these five areas is a set of core activities that should be accomplished in carry- 
ing out the school climate improvement process (Appendix A, page 29). West Virginia enlists a 
cadre of school climate specialists (SCSs), who were trained to mentor school-based teams and 
school leaders through the school climate improvement process. The SCSs’ primary role was to 
facilitate the strategic steps of the WVMPSC among the S3 intervention schools. 

As part of the S3 grant, participating states must calculate a School Safety Score each 
year for every school funded. In support of school climate measurement, the U.S. Department of 
Education, Office of Safe and Healthy Students put forth a model based on a synthesis of availa- 
ble research and expert and stakeholder opinion (National Center on Safe Supportive Learning 
Environments, n.d.). In the model, school climate consists of three primary domains, each con- 
sisting of corresponding subdomains (in italics) as follows: 

• Engagement— the quality of relationships, including respect for diversity, among stu- 
dents, staff and families; the level of school participation and involvement by families, 
staff, and students in school activities; and efforts by schools to connect with the larger 
community. 

• Safety— the physical and emotional security of the school setting and school-related 
activities as perceived, experienced, and created by students, staff, families, and the 
community. The use and trade of illicit substances in the school setting and during 
school-related activities also is included in this domain. 

• Environment— the physical and mental health supports available that promote student 
wellness, the physical condition of school facilities, the academic environment, and the 
disciplinary tone of the school— i.e., the fairness and adequacy of disciplinary proce- 
dures. 
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Organizing 

Establish a school S3 team. 
Train team on WVMSCP 
components. 

Take ownership and commit 
to the process. 


Monitoring & 
Evaluation 

Review data to monitor 
progress and program 
improvement strategies. 

Establish processes for 
project continuation and 
sustainability. 


Assessment 

Conduct a needs assessment 
with survey and incident 
data. 

Establish behavioral 
expectations. 


Planning & Implementation 


Building Support 

Select, implement, and monitor 


Train staff/community partners 

school-wide interventions. 


on WVMSCP components. 

Define district-level interventions or 


Build awareness of behavioral 

referral systems. 


expectations. 

Increase staff, parent, and community 


Address student behavior in a 

capacity to implement interventions. 

< — 

consistent manner. 


Figure 1. Strategic Steps of the WV Model for Positive School Climate 

The WVMPSC involves a set of strategic steps, including (a) organizing, (b) assessment, (c) building support, 
(d) planning and implementation, and (e) monitoring and evaluation. Each strategic step includes a set of 
core activities that should be accomplished in carrying out the school climate improvement process. Only 
selected core activities are shown here for each strategic step of the model. A complete listing of core 
activities is provided in Appendix A. 

To meet the school safety score requirement set forth for the S3 program, the WVDE Of- 
fice of Research developed the WV School Climate Index (Whisman, 2012a). The index— derived 
from 20 indicators drawing from student and staff survey data and selected discipline incident 
data reported in the West Virginia Education Information System (WVEIS)— provides an overall 
measure of school climate tapping all domains and subdomains in the federal school climate 
measurement model described above. 

S3 implementation began in the 2010-11 school year (SY) during which baseline data col- 
lection for school climate measurement began with 22 intervention schools and 25 control 
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schools 8 . Also that year, intervention schools began mobilizing for implementation (i.e., forming 
school-based S3 planning teams, establishing school-wide buy-in, initiating data-driven plan- 
ning processes, etc.). During SY 2011-12, schools began implementing programmatic interven- 
tions to address school climate issues identified through their Year 1 assessment and planning. 

The initial evaluation plan for the WV S3 program proposed to assess five evaluation 
questions: 

EQi. To what extent do participating schools implement the program with fidelity 
relative to the WVMPSC? 

EQ2 . To what extent do program initiatives improve school climate and culture? 

EQ3. To what extent do participating and nonparticipating schools differ in (a) 
school climate and (b) discipline incident counts? 

EQ4. To what extent do perceptions of school climate and student health knowledge 
predict behavioral incident counts? 

EQ5. To what extent is the School Safety Score related to student achievement? 

This interim report, which primarily addresses EQi, and to some extent EQ2, is intended 
to provide midstream feedback to the WVDE program staff. EQ3 and EQ4 are considered more 
long-term outcomes to be assessed in the final year of the grant. EQ5 was addressed for the 
baseline year of school climate measurement (see Whisman 2012b). 

For EQi, implementation fidelity in the S3 project refers to how well schools adhered to 
the strategic steps and core activities framed by the WVMPSC— not how well they implemented 
programmatic interventions. Thus the relevant null hypothesis regarding implementation fideli- 
ty is that schools are not implementing S3 grant activities in a manner consistent with the 
WVMPSC; the alternative is that they are. For measurement purposes, however, what is report- 
ed herein is the extent to which fidelity was met. 

The relevant null hypothesis for EQ2 is that no school climate improvements occurred in 
intervention schools between 2011 and 2012; the alternative is that school climate improve- 
ments did, indeed, occur. 


Evidence-Based Implementation 

The National Implementation Research Network (NIRN) defines implementation as “a 
specified set of activities designed to put into practice an activity or program of known dimen- 
sions” (NIRN, n.d.a). NIRN also describes implementation as a developmental process that oc- 
curs in discernible stages: initial exploration where readiness for implementation is assessed; 
installation where capacity building processes (e.g., staff selection and training) are initiated 
and resources are assembled to support implementation; initial implementation, an awkward 
transitional stage where new activities are beginning to be put into place and former ways of do- 


8 Initially, there were 23 intervention schools and 25 control schools; however, after Year 1 of the 
grant, two intervention schools were merged and thereafter treated as one. The 25 control schools repre- 
sent all other high schools located in the same county district as intervention schools. 
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ing business may be adapted or abandoned altogether; to full implementation where capacity 
has been built and organizational supports are sufficiently internalized into the day-to-day oper- 
ations of the organization (NIRN, n.d.b). 

In the context of the WV S3 project, the set of activities referred to in the implementation 
definition (above) fall within the strategic steps of the WVMPSC. Furthermore, the strategic 
steps and core activities tend to follow a developmental path coinciding with the implementa- 
tion stages described by NIRN (n.d.b). Core activities within the organizing, assessment, and 
building support strategic steps align mostly with the exploration and installation stages of im- 
plementation (Table 11 on page 29). In these steps, S3 schools assemble and train teams to lead 
the implementation process, begin building relationships and securing commitment and sup- 
port, conduct self-assessments in the form of environmental scans and SWOT analyses to evalu- 
ate organizational capacity for S3, and initiate data-driven planning processes and goal setting. 
As one might expect, it is in the planning and implementation step that schools enter into the 
initial implementation stage, and as staff progressively internalize new practices they move to 
full implementation of programmatic interventions. Monitoring and evaluation, considered es- 
sential elements of full implementation, inform course corrections and sustainability. 


In addition to 
progressing through 
the described stages, 
well implemented 
interventions tend to 
share common com- 
ponents— or drivers— 
that support the ef- 
fective use of inter- 
ventions to produce 
positive outcomes 
(Fixsen & Blase, 
2008). Drivers fall 
into three categories 
(Figure 2). Compe- 
tency drivers have to 
do with building or- 
ganizational and staff 
capacity to be effec- 
tive in implementa- 
tion. In the context of 
S3 this is done by as- 
sembling and train- 
ing teams with staff 
receptive to taking on 
the tasks necessary to 
carry out the project 
effectively, and by 
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Figure 2. Evidence Based Implementation Drivers 

Well implemented interventions tend to share common components— or 
drivers— that support the effective use of interventions to produce positive 
outcomes. Drivers fall into three categories: competency, organization, and 
leadership. Figure fromFixsen, D. L., & Blase, K. A. (2008). Drivers framework. 
Chapel Hill, NC: The National Implementation Research Network, Frank Porter 
graham Child Development Institute, University of North Carolina. Used with 
permission. 
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providing ongoing supports in the form of training and coaching (technical assistance) to teams 
and school staff in general (see Table ll on page 29). Here the SCSs play an important role as 
well. 


The WVMPSC, and S3 for that matter, were intentionally designed to be data driven ini- 
tiatives. As such, organizational drivers are abundantly represented among the strategic steps 
and corresponding core activities (see Table 11, page 29). This is especially true with regard to 
the application of decision-support data systems for selecting, implementing, and monitoring 
school level programmatic interventions, and for leveraging systems intervention drivers to fa- 
cilitate the internalization and sustainability of S3 practices into the day-to-day practices of the 
school. 


Leadership drivers can be technical in nature and take the form of good management 
practices in initiating and integrating new interventions and in dealing with known barriers. 
Leadership also can be adaptive in nature and take the form of providing transformational guid- 
ance and effectively responding and adapting to unforeseen challenges. Leadership drivers, both 
technical and adaptive, come into play in S3 with regard to the role and involvement of district 
S3 contacts with their respective schools, the leadership characteristics of school principals, and 
to some extent, school based teams as they build support and commitment among colleagues, 
students, parents, and community partners. Leadership drivers were not as explicitly noted as 
competency and organizational drivers in the crosswalk of the WVMPSC. They are however im- 
portant and are thought to be essential components threaded throughout the process. 


West Virginia Safe and Supportive Schools Project / 5 


6 



Methods 


As mentioned earlier, this report is limited to the first two years of the 4 -year evaluation 
and is focused on two evaluation questions; accordingly, methods described here are for EQl 
and EQ2 only. 


EQl. Implementation Fidelity 

To assess the degree of adherence to the West Virginia Model for Positive School Climate 
(WVMPSC), or fidelity, we developed 4-point rubrics for each core activity (Appendix B, page 
31). The rubric rating scales ranged from missing to strong fidelity, but also included additional 
response options of too early to tell and don’t know. Two respondent groups complete the im- 
plementation fidelity assessments annually, toward the end of each academic year: the cadre of 
school climate specialists (SCSs) and school-based leadership teams in each Safe and Supportive 
Schools (S3) intervention school. SCSs submit one assessment response for each school they 
serve; members of the S3 teams collaborate to complete one rubric for their school, representing 
their consensus view of progress made during the year. 

To gauge change in implementation fidelity over time, we compared ratings between 
school years for each group. Fidelity assessment data were available from both groups (S3 teams 
and SCSs) for both project years (2011 and 2012) for only 15 intervention schools. Since looking 
at change over time represented an analysis of two dependent paired samples but among only 15 
schools, a distribution free approach was warranted. The Wilcoxon sign rank test was used. 

We assumed that S3 teams and SCSs would have different opinions about the extent to 
which schools were implementing with fidelity— especially considering SCSs have more experi- 
ence with the WVMPSC and all but one of the SCSs currently assist multiple schools (up to 
eight). Therefore, SCSs should have a broader frame of reference against which to judge imple- 
mentation progress. Tests for differences between SCSs and school-based S3 teams amounted to 
a comparison of independent samples, so we used the Mann- Whitney U test. 

EQ2. School Climate Improvement 

Two approaches were taken to assess school climate improvement for purposes of this 
study. First, we compared overall school climate index scores for S3 intervention schools be- 
tween the 2010-11 and the 2011-12 school years to assess change over time. For this purpose suf- 
ficient data were available to calculate the index for 21 intervention schools and 20 non- 
intervention schools during the baseline 2010-11 school year. Index scores were computed using 
student and staff survey data and selected discipline incident data collected from both interven- 
tion and nonintervention comparison schools in the baseline year, with each school’s index 
score calculated relative to the aggregate of all schools combined. The nonintervention schools, 
however, were required to participate in data collection only in the baseline 2010-11 school year, 
but not again until the 2013-14 school year— the final year of the S3 grant program. We calculat- 
ed annual changes in the index for intervention schools relative to data from the non- 
intervention schools held constant at the 2010-11 baseline levels. 
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In interpreting the results it is important to note that the index was calculated as a 9- 
point stanine scale score from an asset-based perspective— i.e., higher index scores correspond 
to more favorable school climate conditions. Accordingly, increases in scores over time should 
be interpreted to indicate improvements in school conditions. As with the fidelity assessment 
described above, the unit of analysis was the school. Because we are comparing the same schools 
over 2 years— i.e., two dependent paired samples— and only 21 schools were included, it seemed 
prudent to assume a distribution free approach was appropriate. The corresponding nonpara- 
metric procedure was the Wilcoxon sign rank test, which we replicated for each of the 20 indica- 
tors making up the overall index 9 . 

Second, we added a question to the spring 2012 administration of the WV School Climate 
Surveys for students and staff. In the question, we listed a series of items corresponding to 20 
indicators contained within the school climate index. We asked respondents to indicate whether 
the items have changed compared to the previous year. Although there are only 20 indicators in 
the school climate index, we used 22 items in the survey in order to split two indicators. Rela- 
tionships at school was split to capture relationships among students and relationships between 
students and staff. Likewise, we split students' avoiding school activities or specific places in 
school to distinguish between school activities on the one hand and places at school on the oth- 
er. The questions included a 5-point response scale, ranging from a lot better to a lot worse, 
with a midpoint response of stayed about the same. Only students and staff who indicated in a 
screening question that they had attended or worked at the school the previous school year were 
presented with these items. 

The chi square statistic was used to test the null hypothesis of no improvements in 
school climate. The initial dataset consisted of 4,537 students and 697 staff responses. Prelimi- 
nary data analysis indicated that most respondents in both groups reported conditions to have 
either improved or stayed the same, whereas few indicated conditions had gotten worse. The 
latter was problematic, especially with regard to staff responses, because the chi square statistic 
becomes unreliable when expected cell counts are less than five— which was the case for several 
items analyzed. To resolve this issue, we collapsed the 5-point response scale to a 3-point scale 
consisting of a little or a lot better, stayed about the same, and a little or a lot worse. Finally, 
representation in the dataset was lopsided in favor of students by a 6.5 to 1 ratio. Although it did 
not necessarily violate the assumptions for the proposed analysis, another step was taken to bal- 
ance the representation of the two groups by selecting a random sample of students in a number 
equal to the 697 staff responses. 

We worked with the assumption that, all things being equal, students and staff would be 
equally likely to think conditions had gotten better, stayed the same, or gotten worse. As such, 
we constructed a dummy dataset with expected cell proportions equally distributed at 33.3% in 
each of the collapsed response categories. Cross-tabulations were then performed separately for 
each group comparing cell proportions observed from the survey data against the expected pro- 
portions. Subsequently, we conducted cross tabulations and chi square tests to compare ob- 
served cell proportions between students and staff for group differences. 


9 See Whisman, 2012a for a more detailed description of the development and application of the 
WV School Climate Index. 
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Ancillary Data Analysis 

When entering into a memorandum of understanding (MOU) with the WVDE, county 
school districts committed to certain obligations relative to the S3 project, including (a) assign- 
ing a central office staff member to serve as contact for the S3 project, (b) providing leadership 
support to the 22 intervention schools, and (c) providing fiscal and management support. How 
much support to provide, and the level of involvement of the district S3 contact, were decisions 
left to the districts and schools. Anecdotally, however, SCSs pointed out that wide variation ex- 
isted among districts regarding support and involvement. To gauge the level of variation, SCSs 
were asked in the 2012 fidelity assessment the extent of participation and support schools re- 
ceived from their respective district contacts. In previous feedback, SCSs suggested that the de- 
gree to which school principals are engaged in and support the S3 project— which also varied 
widely— may have some bearing on S3 team functioning and progress. Consequently, SCSs also 
were asked in the 2012 fidelity assessment to characterize the nature of participation and sup- 
port each school’s principal contributed to the S3 project. 

Additionally, school S3 teams were asked a few questions beyond the fidelity assessment 
rubrics. A critical component of the WVMPSC calls for schools to establish behavioral norms or 
expectations for students and staff, which in turn serve as a basis for selecting and implementing 
programmatic interventions. Teams were asked to list the behavioral norms or behavioral ex- 
pectations the school had defined for students and staff. Space was provided for up to ten open- 
ended responses. The assessment also asked teams the extent to which students and staff at the 
school provided input into establishing the school-wide behavioral norms/expectations. Finally, 
teams were asked to indicate the methods or processes used to teach students and staff about 
the behavior norms or expectations. Responses to the added questions were categorized and 
presented descriptively. 
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EQl. Implementation Fidelity 

Charts showing school-based Safe and Supportive Schools (S3) team and school climate 
specialist (SCS) fidelity assessments of the core activities on the West Virginia Model for Positive 
School Climate (WVMPSC) for 2011 and 2012 are presented in Appendix C (page 39). Based 
simply on the observed modal responses from the charts, there were three core activities for 
which both groups indicated strong fidelity with the components of the WVMPSC, and they held 
that opinion in both 2011 and 2012. Two of these activities, establishing and training the school- 
based team and establishing ownership and commitment to the S3 initiative, fell in the organiz- 
ing strategic step of the WVMSCP. The third, in the assessment strategic step, had to do with 
conducting a strengths, weaknesses, opportunities, and threats (SWOT) analysis to examine the 
school’s internal operations relative to the S3 project. Note that these items were indicated as 
being implemented with strong fidelity based on modal responses— i.e. the rating most frequent- 
ly selected by both the S3 teams and SCSs. There was variation in their ratings, however, sug- 
gesting room for improvement. 

All other core activities generally appeared to be rated at lower levels of fidelity by both 
groups, particularly for the 2011 ratings. When looking at assessment ratings within groups, 
Wilcoxon signed ranks tests confirmed that the fidelity assessments in 2012 tended to be higher 
than those in the previous year. Although there were no improvements on activities in the or- 
ganizing strategic step among either group, the ratings of S3 teams showed improvement on 
three of six core activities under the assessment strategic step, and on all activities under the 
building support and planning strategic steps (Table 17, page 47). They also indicated statisti- 
cally significant increases in implementation fidelity on one core activity under the monitoring 
and evaluation step— that the team reviewed or analyzed data on an ongoing basis to make pro- 
ject adjustments. The ratings of the SCSs showed significant increases on every core activity un- 
der the Assessment, Building Support, Planning, and Monitoring and Evaluation strategic 
steps between 2011 and 2012 (Table 18, page 48). Across most core activities the observed im- 
provements consisted of moving from being altogether missing or implemented with weak fidel- 
ity in 2011 to being implemented at weak or moderate fidelity in 2012. 

When comparing the 2011 assessment ratings between S3 teams and SCSs, statistically 
significant differences were observed on only three core activities (Table 19 on page 49). For all 
three, S3 teams rated their implementation at higher levels compared to SCSs. The activities in- 
cluded two under the building support strategic step— informing parents and community part- 
ners about the S3 initiative and securing their commitment, and building understanding of S3 
behavioral norms among school-level professional and support staff. The third activity had to do 
with establishing plans and processes for ensuring long-term project continuation and sustaina- 
bility under the monitoring and evaluation strategic step. 

Comparing the two groups on the 2012 assessment ratings, however, revealed differ- 
ences on seven core activities (Table 20 on page 50) and in all cases school-based S3 teams rated 
their implementation of activities at greater levels of fidelity than the SCSs. The first had to do 
with establishing ownership and commitment under the organizing strategic step. Collectively 
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based on median scores, this activity was rated as being implemented with fairly strong fidelity 
by both groups, yet the chart of assessment ratings for this activity at the bottom of page 39 
shows that all school S3 teams indicated implementation with strong fidelity, whereas SCSs rat- 
ed only 10 of 15 schools at strong fidelity. They also differed on three of five activities under the 
building support strategic step: informing and securing commitment of parents and community 
partners; building understanding of behavioral norms among school -level professional and sup- 
port staff; and defining school-wide procedural expectations for addressing student behavior. 


Under the planning strategic step, differences were found on two core activities— both 
having to do with using assessment results to inform the selection of interventions. The first 
gauges the use of comprehensive assessment results to identify factors contributing to school 
climate problems, set priorities, or plan activities; and the second dealt with using contributing 
factors identified in the assessment process to select appropriate interventions. This finding 
points to a potential problem in terms of achieving the best possible outcomes in school climate 
improvement efforts; if schools are selecting interventions on the basis of inadequate assess- 
ments of factors contributing to problematic school climate issues, then the likelihood of achiev- 
ing better outcomes may be diminished. Finally, under the monitoring and evaluation strategic 
step, differences were found for one activity— reviewing or analyzing data on an ongoing basis to 
make project adjustments. 


In the 2012 fidelity assessment complet- 
ed by SCSs, an additional question about im- 
plementation fidelity relative to the WVMPSC 
was included to obtain respondents’ overall view 
of schools’ implementation progress. SCSs re- 
ported none of the 22 intervention schools had 
done an excellent job in carrying out the 
WVMPSC, yet they did indicate that 12 (55%) 
were implementing at above average levels 
(Table 1). Of the remaining schools, they indi- 
cated nine (41%) were implementing at average 
levels, and only one at below average levels. 


Table 1. School Climate Specialists' Overall 
Assessment of Fidelity With Which 
Schools had Implemented the Positive 
School Climate Model S3 Grant Project 



Frequency 

Percent 

Below Average 

1 

4.5 

Average 

9 

40.9 

Above Average 

12 

54.5 

Excellent 

0 

0.0 

Total 

22 

100.0 


EQ2. School Climate Improvement 

School Climate Index comparisons 

Sixteen of 21 intervention schools experienced increases in their overall school climate 
index between 2010-11 and 2011-12 (Table 2). Across all schools, the median overall index score 
increased from 4.75 at baseline year to 5.17 the following year. The median annual increase in 
the index scores was 0.44 points on the 9-point stanine scale; however, three schools (Interven- 
tion Schools 14, 18, and 21) saw comparatively large jumps of 1.37 to 1.73 points. The Wilcoxon 
signed-rank test bears out that the observed improvement among intervention schools was sta- 
tistically significant (z = -2.92, p = 0.005). The corresponding effect size (r = -0.45) suggests a 
medium to large increase in index score over the 2 years. 
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Table 2. Overall School Climate Index Scores for S3 

Intervention Schools, 2011 and 2012 School Years 


School 

2011 

2012 

Annual 

change 

Intervention School 1 

4.57 

4.63 

.06 

Intervention School 2 

5.59 

6.36 

.77 

Intervention School 3 

3.11 

4.03 

.92 

Intervention School 4 

4.62 

5.55 

.93 

Intervention School 5 

3.79 

4.22 

.43 

Intervention School 6 

5.33 

4.53 

-.80 

Intervention School 7 

5.86 

6.30 

.44 

Intervention School 8 

5.72 

5.94 

.22 

Intervention School 9 

5.85 

6.29 

.44 

Intervention School 10 

5.04 

4.63 

-.41 

Intervention School 11 

5.48 

5.47 

-.01 

Intervention School 12 

4.75 

4.63 

-.12 

Intervention School 13 

5.42 

5.17 

-.25 

Intervention School 14 

4.46 

6.19 

1.73 

Intervention School 15 

4.73 

5.51 

.78 

Intervention School 16 

3.32 

4.31 

.99 

Intervention School 17 

4.48 

4.67 

.19 

Intervention School 18 

3.33 

4.82 

1.49 

Intervention School 19 

5.10 

5.90 

.80 

Intervention School 20 

4.71 

5.14 

.43 

Intervention School 21 

5.47 

6.84 

1.37 


tobacco/alcohol/drugs on school property ( z = -2.40, p - 
lower (-0.32 to -0.40), but indicated medium size effects. 


Replicating the analysis on 
each of the 20 indicators making up 
the overall school climate index re- 
vealed statistically significant im- 
provements on eight of the 20 indi- 
cators (Table 3). Three of the indica- 
tors fall in the engagement domain 
of the school climate measurement 
model, and include relationships at 
school (z = -3.33, p = 0.001), respect 
for racial, ethnic, or cultural diver- 
sity (z = -3.09, p - 0.002), and 
meaningful participation in school 
(z = -3-35, P = 0.001). Effect sizes 
(r) ranged from -0.48 to -0.52, sug- 
gesting moderate to strong increases 
in scores. 

Three indicators in the safe- 
ty domain showed significant im- 
provements. In the area of physical 
safety physical fights on school 
property (z = -2.58, p = 0.01) and 
students' carrying weapons on 
school property (z = -2.90, p = 
0.04) improved. Students' use of 
0.02) also improved. Effect sizes were 


Significant improvements were observed on two indicators in the environment school 
climate domain: supportive academic environment (z = -3.37, p - 0.001) and discipline prob- 
lems reported at school (z = -3.42, p = 0.001). Strong effect sizes were observed for these indica- 
tors. 


WV School Climate Survey results for students and staff 

Testing against the expectation that one third (33.3%) of respondents would say that the 
22 items corresponding to school climate conditions had gotten better, stayed the same, or got- 
ten worse, respectively, all cross-tabulations were statistically significant among students (x 2 = 
30.3 to 206.6, p<o.oooi) and staff (x 2 = 61.3 to 274.3, p<o.oooi) 10 . This depended, however, on 
the response category. Fairly low and statistically significant percentages of both students and 
staff reported that conditions had deteriorated; across all 22 items an average of only 11% of 
students and 5% of staff indicated things had gotten a little or a lot worse compared to the previ- 


10 Complete cross-tabulation and chi square results are provided for students in Table 21, 
page 51 and for staff in Table 22, page 53). 
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ous year (Table 4 and Table 5). On the opposite end of the response scale however, students 
were much less likely than staff to indicate things had gotten a little or a lot better, and much 
more likely to indicate things had stayed about the same. On only four of the 22 items did the 
proportion of students exceed the expectation by a statistically significant margin. These includ- 
ed relationships among students, meaningful opportunities for participation in school, physi- 
cal fights on school property, and safety and security measures observed at school (Table 4). 
Another statistically significant cell proportion was found among students ( students' use of to- 
bacco/alcohol/drugs on school property), which at a cell proportion of 26.1% was in the wrong 
direction from the expectation. Across all 22 items an average of 55% of students indicated con- 
ditions were about the same as the previous year, whereas an average of 34% indicated condi- 
tions had gotten better. 


Table 3. Nonparametric Tests for Changes in School Climate Indicator Index Scores Among S3 Intervention 
Schools From 2011 to 2012 





2011 

2012 




School climate domain 

Indicator 

median 

median 

z 

P 

r 

4-> 

c 

CD 

Relationships 

Relationships at school 

4.60 

5.40 

-3.33 

0.001 

-0.51 

E 

CD 

QO 

n: 

CUD 

c 

LU 

Respect for 

Diversity 

Respect for racial, ethnic, or cultural diversity 

4.75 

5.75 

-3.09 

0.002 

-0.48 

Participation 

Meaningful participation in school 

4.67 

6.33 

-3.35 

0.001 

-0.52 


Emotional Safety 

Bullying at school and cyber-bullying anywhere 

4.50 

5.25 

-1.11 

0.26 




Students avoiding school activities or specific 
places in school 

4.50 

5.00 

-0.75 

0.45 




Students being called hate-related words / 
seeing hate-related graffiti 

5.00 

4.50 

-0.63 

0.53 




Students perceptions of personal safety at 
school 

5.00 

5.00 

-1.86 

0.06 



Physical Safety 

Physical fights on school property 

4.50 

5.50 

-2.58 

0.01 

-0.40 

> 

-M 

CD 


Students carrying weapons on school property 

5.00 

5.00 

-2.09 

0.04 

-0.32 

n: 

on 


Students reports of safety/security measures 
observed at school 

5.00 

6.00 

-0.99 

0.32 




Teachers threatened with injury or physically 
attacked by students 

6.00 

7.00 

-1.40 

0.16 




Threats and injuries with weapons on school 
property 

5.00 

5.00 

-0.58 

0.56 




Violent and other crime incidents at school 

6.00 

5.00 

-1.27 

0.21 



Substance Abuse 

Students' use of tobacco/alcohol/drugs on 
school property 

4.67 

5.67 

-2.40 

0.02 

-0.37 


Physical 

Teachers' and students' reports on school 

4.50 

5.00 

-1.54 

0.12 



Environment 

conditions 






+-> 

c 

QJ 

Academic 

Supportive academic environment 

4.88 

5.50 

-3.37 

0.001 

-0.52 

E 

Environment 







0 

Well-Being 

Students' physical or mental health 

4.60 

5.00 

-1.19 

0.23 


’> 

C 

LU 

Disciplinary 

Discipline problems reported at school 

4.50 

5.25 

-3.42 

0.001 

-0.53 


Environment 

Reports of gangs at school 

5.50 

5.50 

-0.13 

0.89 




Serious disciplinary actions taken by schools 

6.00 

5.00 

-1.31 

0.19 
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Table 4. Student Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School 
Years 


Item 

Percent a 

little or a 

lot better 

Percent 

about the 

same 

Percent a 

little or a 

lot worse 

Average 

34.5 

55.0 

10.6 

Relationships among students 

45.3 a 

43.7 a 

11.1 a 

Relationships among students and staff 

37.6 

50.4 a 

12.1 a 

Respect for racial, ethnic, or cultural diversity 

32.4 

59.4 a 

8.2 3 

Meaningful opportunities for participation in school 

40.2 a 

51.2 a 

8.7 a 

Bullying at school 

35.8 

48.1 a 

16.1 3 

Students perceptions of personal safety at school 

32.9 

59.1 a 

8.0 3 

Students avoiding school activities for fear of being harmed 

33.2 

59.5 a 

7.3 3 

Students avoiding specific places in school for fear of being harmed 

32.5 

60.6 a 

7.0 3 

Students being called hate-related words and seeing hate-related graffiti 

30.9 

57.7 a 

11.4 3 

Physical fights on school property 

41.9 a 

45.1 a 

13.1 3 

Students carrying weapons on school property 

36.1 

57.2 a 

6.7 3 

Safety and security measures observed at school 

38.8 a 

54.4 a 

6.9 3 

Teachers threatened with injury or physical attack by students 

33.6 

61.5 a 

4.8 3 

Threats and injuries with weapons on school property 

31.9 

60.6 a 

7.5 3 

Violent and other crime incidents at school 

32.3 

61.5 a 

6.3 3 

Gang activity at school 

31.0 

62.5 3 

6.4 3 

Students use of tobacco/alcohol/drugs on school property 

26.1 a 

47.7 a 

26.2 3 

School conditions (clean and in good condition) 

31.8 

51.6 a 

16.6 3 

The school being a supportive academic environment 

36.3 

55.7 3 

8.0 3 

Students' physical or mental health 

30.2 

59.2 3 

10.6 3 

Discipline problems reported at school (disruptive behavior or cutting 

31.9 

51.5 a 

16.5 3 

classes/truancy) 




Disciplinary actions taken by schools (suspensions, detention, etc.) 

35.2 

51.7 3 

13.1 3 


a Indicates statistically significant difference from the expected cell proportion of 33.3%. 

Conversely, staff were much more likely to indicate conditions had improved relative to 
the expectation. Across all 22 items an average 51% of staff indicated things had gotten a little or 
a lot better and on only two items did they fail to beat the expected cell proportion of 33.3%. 
These included students' use of tobacco/alcohol/drugs on school property and respect for ra- 
cial, ethnic, or cultural diversity (Table 5). 

In the results above, student and staff responses were compared against an expectation 
that equal proportions of each group would report that conditions had improved, stayed the 
same, or declined. When comparing students and staff against each other, differences similar to 
those reported above were observed. On all 22 items the proportion of staff indicating things 
had gotten a little or a lot better exceeded the proportion of students having the same opinion by 
significant margins (see Table 23 on page 55). Furthermore, on 16 of 22 items statistically great- 
er proportions of students reported conditions had stayed about the same and, although at lower 
percentages, on 19 items a statistically greater proportion of students reported things had gotten 
a little or a lot worse. All told, these findings reveal fairly sharp differences between students 
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and staff in their perception of school climate improvement between the 2011 and 2012 school 
years. 


Table 5. Staff Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School 
Years 


Item 

Percent a 

little or a lot 

better 

Percent 

about the 

same 

Percent 

a little or a 

lot worse 

Average 

51.0 

44.4 

4.6 

Relationships among students 

58.3 a 

39.1 a 

2.6 a 

Relationships among students and staff 

60.9 a 

33.9 

5.2 a 

Respect for racial, ethnic, or cultural diversity 

37.9 

60.5 a 

1.6 a 

Meaningful opportunities for participation in school 

64.7 a 

33.8 

1.4 a 

Bullying at school 

48.3 a 

44.8 a 

6.8 a 

Students perceptions of personal safety at school 

56.5 a 

41.3 a 

2.2 a 

Students avoiding school activities for fear of being harmed 

50.4 a 

48.3 a 

1.3 a 

Students avoiding specific places in school for fear of being harmed 

49.3 a 

49.0 a 

1.7 a 

Students being called hate-related words and seeing hate-related graffiti 

50.6 a 

45.5 a 

3.9 a 

Physical fights on school property 

63.4 a 

32.7 

3.9 a 

Students' carrying weapons on school property 

47.7 a 

51.3 a 

1.0 a 

Safety and security measures observed at school 

61.0 a 

37.1 

1.9 a 

Teachers threatened with injury or physical attack by students 

46.1 a 

50.4 a 

3.5 a 

Threats and injuries with weapons on school property 

46.0 a 

53.1 a 

0.9 a 

Violent and other crime incidents at school 

49.9 a 

49.4 a 

0.7 a 

Gang activity at school 

41.2 a 

58.3 a 

0.6 a 

Students use of tobacco/alcohol/drugs on school property 

33.4 

49.9 a 

16.8 a 

School conditions (clean and in good condition) 

46.2 a 

42.8 a 

11.0 a 

The school being a supportive academic environment 

62.7 a 

33.5 

3.8 a 

Students' physical or mental health 

42.4 a 

53.1 a 

4.5 a 

Discipline problems reported at school (disruptive behavior or cutting 

49.8 a 

36.3 

13.9 a 

classes/truancy) 




Disciplinary actions taken by schools (suspensions, detention, etc.) 

55.7 a 

33.4 

10.9 a 


a Indicates statistically significant difference from the expected cell proportion of 33.3%. 


Results from Ancillary Data Analysis 

Role and involvement of district S3 contacts and school principals 

SCSs reported that in eight intervention schools (36%) the district coordinators have 
been very involved in supporting the S3 project by attending meetings, contributing to planning 
processes, or other forms of support (Table 6). There also were five schools (23%) where district 
coordinators were reported to have been involved but at more modest levels. Conversely, at 
eight schools the district coordinators served only as the county contact for the grant but other- 
wise were not involved in the project, according to SCSs. One SCS response indicated district 
contact involvement different from the options provided in the assessment, but the SCS’s de- 
scription of that involvement indicated it was in the form of fiscal management and support as 
noted among the expectations listed in the MOU. 
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Table 6. School Climate Specialist Assessment of Nature of the District Coordinator's Participation and 

Support for the S3 Project 

Frequency Percent 

Total 

22 

100.0 

The District Coordinator serves only as the county contact for the S3 Grant, but 
otherwise is not involved in project planning and implementation at this school. 

8 

36.4 

In addition to serving as the county contact, the District Coordinator has some 
involvement in project planning and implementation at this school (i.e., attends a few 
team meetings, communicates S3 expectations, etc.). 

5 

22.7 

The District Coordinator has been very involved in project planning and 
implementation at this school (i.e., frequently attends team meetings, contributes 
substantially to planning, provides implementation guidance, etc.). 

8 

36.4 

Don't Know 

0 

0 

Other 

1 

4.5 


In a large majority of intervention schools— 15 (68%) of the 22— principals were reported 
to be very involved in the S3 project by frequently attending team meetings, contributing to 
planning processes and providing implementation guidance (Table 7). In another four (18%), 
school principals were reported to be involved in the project, but at lower levels. For only two 
schools were principals reported to be largely uninvolved. 


Table 7. School Climate Specialist Assessment of Nature of the School Principal Participation and Support for 
the S3 Project 



Frequency 

Percent 

Total 

22 

100.0 

The Principal for the most part is not involved in project planning and implementation 
at this school. 

2 

9.1 

The Principal has some involvement in project planning and implementation at this 
school (i.e., attends a few team meetings, communicates S3 expectations, etc.). 

4 

18.2 

The Principal has been very involved in project planning and implementation at this 
school (i.e., frequently attends team meetings, contributes substantially to planning, 
provides implementation guidance, etc.). 

15 

68.2 

Don't Know 

0 

0 

Other 

1 

4.5 


School defined behavior expectations 


Although no specific guidance is provided in the WVMPSC on setting an appropriate 
number of behavioral norms or expectations for a school, from a similar framework— School- 
Wide Positive Behavior Intervention Supports (SWPBIS)— three to five clearly stated 
norms/ expectations maybe optimal (Algozzine, et al., 2010). Behavior norms or expectations 
also should be stated clearly and in ways that promote and support appropriate behavior among 
students and staff. Among other criteria, expectations should be 1-3 words in length each, posi- 
tively stated, comprehensive (apply to all students and staff, and all settings); and mutually ex- 
clusive. 


Twenty-one of the 22 intervention schools responded with a total of 105 behavior norms 
or expectations established for the schools through the S3 project. The number of behavior ex- 
pectations listed by the S3 intervention schools ranged from two to 10 per school. Most schools, 
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13 (62%) of those reporting, listed five or fewer in alignment with the recommendation. The re- 
maining 8 schools having set more than five may benefit from revisiting the issue and refining 
their behavior expectations. 

It was not always clear the explicit behaviors the expectations were intended to reinforce; 
however, it was possible from an inspection of the responses to group them into categories. 
Although some subjectivity entered into the categorization process, a fairly high level of com- 
monality was observed in the categories into which behavior expectations fell. For example, 16 
(76%) of the 21 schools listed behavioral expectations having to do with being respectful or hav- 
ing respect for self and others (Table 8). Thirteen schools (62%) articulated expectations related 
to timeliness, presumably targeting issues of tardiness or attendance identified in their needs 
assessment processes. Most of the time these were as simply stated as “be on time,” whereas a 
few were more specifically stated— “attend school every day.” Some schools appeared to have 
embedded expected behaviors within slogans, for example “Rigor, Relevance, Relationships,” 
although it is not always clear from the slogans precisely the behaviors targeted. Other catego- 
ries into which behavior expectations fell included, in a descending number of schools, self- 
control, zero tolerance, preparedness, responsibility, kindness, engagement, staff practices, be- 
ing on task, and safety (Table 8). 

Table 8. Categories of Behavior Norms or Expectations Established by S3 Intervention Schools 


Number of Percent of 
behavior behavior 



Number of 

Percent of 

norms/ 

norms/ 

Category 

Examples 

schools 

schools expectations expectations 

Respectfulness 

Be Respectful; Respect is key; Respect 
yourself and others 

16 

76.2 

18 

17.1 

Timeliness 

Attend school every day; Be on time; Be 
present 

13 

61.9 

16 

15.2 

Slogan 

Dare to Dream; Rigor, Relevance, 

Relationships; Bell to bell will serve you 
well 

6 

28.6 

11 

10.5 

Self-Control 

Adjust your actions; Beware of wrong 
choices; Exercise good judgment 

5 

23.8 

11 

10.5 

Zero Tolerance 

Violence is not tolerated at our school; 

Zero tolerance for bullying 

7 

33.3 

10 

9.5 

Preparedness 

Be prepared; Be ready to learn; Come to 
class prepared to learn 

8 

38.1 

7 

6.7 

Responsibility 

Be Responsible; Learning is a 
responsibility 

6 

28.6 

6 

5.7 

Kindness 

Be polite; Practice kindness; Speak with 
kindness 

5 

23.8 

5 

4.8 

Engagement 

Be productive; Give your best effort 

4 

19.0 

4 

3.8 

Staff Practices 

Greet students at the door 

4 

19.0 

3 

2.9 

On Task 

Initiative; Learning 

2 

9.5 

3 

2.9 

Safety 

Be Safe 

3 

14.3 

2 

1.9 

Other 

Be honest; Be a good citizen; Practice 

2 

9.5 

9 

8.6 


unity 
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It is also noteworthy that the total number of behavior expectations in each category in 
Table 8 often exceeded the number of schools expressing the expectation, indicating that to 
some extent, schools expressed their expectations in certain areas in multiple ways. For exam- 
ple, there were 16 behavior expectations under the timeliness category but for only 13 schools 
(Table 8). One school listed both “be on time” and “attend school regularly” as expectations for 
students. The school appears to be drawing a distinction between tardiness on the one hand and 
full blown attendance on the other, an understandable stance where both have been identified as 
issues in need of intervention. This example also meets with the recommendation that expecta- 
tions be mutually exclusive. 

For the most part, the behavior expectations provided by the S3 intervention schools 
were clearly and positively stated. Some, however, were not. For example, some were directives 
or pronouncements of policy that were far too wordy and negatively stated, for example, “Be 
where you are supposed to be when you are supposed to be there,” or “You may not have more 
than 10 unexcused absences in any one class in a semester.” Others— for example, “Do what you 
are supposed to be doing”— are ambiguous and appear to provide little information about what 
students and staff are indeed supposed to be doing. In such cases, benefit may be derived from 
reframing the expectations in more direct and positive ways so that students and staff clearly 
know what is expected. 

It appears that students and staff as 
a whole were provided the opportunity to 
have input into establishing school-wide 
behavioral expectations among all 22 inter- 
vention schools (Table 9). Although the re- 
sponse categories provided in the assess- 
ment were not precisely defined, S3 teams 
at half of the schools indicated a moderate 
level of student and staff input in the pro- 
cess, while for another seven (32%), stu- 
dents and staff provided input to a major 
extent. These examples describe how input was obtained: 

The S3 team used the survey data from Spring 2011 to set norms. The norms 
[then] were agreed upon by a majority of the staff. 

We formed a student advisory group which had input [in establishing 
norms/expectations] as well as helped in creating posters and shirts. 

The S3 team discussed with students and staff what we expected, and as a group 
we established the [school slogan]. 


Table 9. Extent to Which Students and Staff at S3 


Intervention Schools Provided Input Into 
Establishing School-Wide Behavioral 
Norms/Expectations 


Number of schools 

Percent 

Not at all 

0 

0% 

To a minor extent 

4 

18.2% 

To a moderate extent 

11 

50.0% 

To a major extent 

7 

31.8% 

Total 

22 

100.0% 
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Table 10. 


Approaches Used by S3 Intervention Schools to 
Teach Behavior Norms/Expectations to Students 
and Staff 


A combination of whole- 
school, classroom-based, and one-on- 
one approaches may be used to effec- 
tively teach and reinforce behavior 
expectations. S3 intervention schools 
appeared to tend toward more whole- 
school approaches (Table 10). All 22 
schools indicated that whole-school 
assemblies were used, followed by 
morning announcements in 19 (86%) 
schools. Nineteen schools indicated 
that reteaching behavior expectations 
happens when correcting inappropri- 
ate behavior— a one-on-one interac- 
tion presumably occurring at the time of the behavior. Fewer than half of the schools indicated 
formal curricula or programs, staff-developed or online lesson plans, or skits or student- 
developed videos or messages were used to teach expectations. 



Number 

Percent 

Formal evidence-based curricula or program 

7 

31.8 

Student developed videos or messages 

5 

22.7 

Staff developed lesson plans 

9 

40.9 

Online lesson plans 

3 

13.6 

Role playing or skits 

8 

36.4 

School assemblies 

22 

100.0 

Morning announcements 

19 

86.4 

Reteaching when correcting inappropriate 
behavior 

19 

86.4 
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Implementation 

Overall the 15 (of 22) intervention schools for which fidelity assessment data were avail- 
able in both 2011 and 2012 appeared to have done well in implementing core activities that align 
with the exploration and installation stages of implementation. Moreover, competency and or- 
ganization implementation drivers tended to be largely in play during the first 2 years of the 
project as school-based teams were established, school climate specialists (SCSs) provided train- 
ing and developed coaching relationships with school teams, and data systems to support deci- 
sions were employed through core activities involving environmental scans of school conditions 
and SWOT analyses. The use of other relevant data sources (e.g., school climate survey data) 
was initiated and sustained over the 2 years to support school improvement purposes. Across 
most core activities— aligned with all stages of implementation— improvements were made, 
moving from being altogether missing or implemented with weak fidelity in 2011, to being im- 
plemented at weak to moderate fidelity in 2012. 

Both school-based Safe and Supportive Schools (S3) teams and SCSs indicated marked 
improvements relative to the strategic steps of the West Virginia Model for Positive School Cli- 
mate (WVMPSC) over the 2 years. SCSs tended to be more guarded in their assessments, how- 
ever; for some core activities they indicated schools’ implementation fidelity to be at lower levels 
than the school S3 teams rated themselves. This was not entirely unexpected because of the 
broader frame of reference against which SCSs may view consistency with the WVMPSC. There 
appeared to be a greater degree of disagreement between teams and SCSs in 2012 than in 2011. 
In 2012 statistically significant differences were found on seven WVMPSC core activities com- 
pared to only three in 2011, on two of which they differed in both years: (a) informing parents 
and community partners about the S3 initiative and securing their commitment and (b) building 
understanding of S3 behavioral norms among school staff. The first of these is a systemic chal- 
lenge throughout the entire school system, whether involved in S3 or not. Although there are 
exceptions, it remains difficult to facilitate high levels of parent and community involvement in 
school initiatives. 

The latter core activity is of critical concern (i.e., building understanding of S3 behavioral 
norms among school staff) because successful school climate improvement relies heavily on the 
awareness and understanding of expectations for appropriate behaviors, and execution of prac- 
tices that reinforce those expectations in the day-to-day operations of the school. A related activ- 
ity on which the two groups differed in 2012 was defining school-wide procedural expectations 
for addressing student behavior. That SCSs assessed these activities at lower levels of fidelity 
than the school teams points to the need for competency-building opportunities for school team 
members and their colleagues, and for SCSs to coach them along. 

The other core activities that SCSs rated at statistically lower levels than did school S3 
teams relate to the use of decision-support data systems. These activities included using assess- 
ment results to (a) identify factors contributing to school climate problems, (b) set priorities or 
plan activities, and (c) select appropriate interventions. Differences also were found on schools’ 
efforts to review or analyze data on an ongoing basis to make project adjustments. As noted ear- 
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lier, it is not enough to simply identify problematic school climate issues. Achieving the best 
possible climate improvement also depends heavily on selecting and implementing interven- 
tions based on thorough assessments of factors contributing to those problematic issues. In- 
creased attention to the exploration of these factors is an area where schools could reap substan- 
tial benefits from their efforts. 

Leadership implementation drivers are not as explicitly described in the strategic steps 
of the WVMPSC as are competency and organization drivers. Two areas where SCSs provided 
limited data were the role and involvement of district S3 contacts and of school principals. SCSs 
reported that for most intervention schools, the district S3 contacts had at least some involve- 
ment in the S3 project at the school level, and for eight schools a fairly high level of involvement 
was noted. For eight other schools, however, district coordinators were reported to have had lit- 
tle involvement beyond serving as the contact for the grant. One could assume that high levels of 
involvement by the district contact would be beneficial to the implementation of the WVMPSC, 
but that assumption has not been validated. There could be good reasons for varying levels of 
involvement. In some cases, district contacts may be confident that schools have the capacity to 
carry out the grant with little assistance, while for other schools they believe the opposite may be 
true and they gauge their involvement accordingly. Nevertheless it is not clear from the limited 
findings in this study if district contact involvement has a positive or negative effect on schools’ 
success. 

In an examination of educational planning processes, Beach and Lindahl (2007) note 
that in modifying a school's climate and culture, school administrators can play a substantial 
role in clarifying and consistently modeling the beliefs and values embodied in any proposed 
changes in the school environment. They can create the conditions in which students and staff 
internalize new practices, such as the behavior expectations established through the WVMSPC. 
Although it is not clear the specific roles of school principals among S3 intervention schools, 
SCSs reported that in a large majority of schools, principals had been very or moderately in- 
volved in the S3 project, which bodes well for the project as a whole. There was a small number 
of schools where principals were reported to be largely uninvolved, and this may pose a substan- 
tial barrier. 


School Climate 

As a group, S3 intervention schools showed significant improvement with medium to 
large effect sizes in school climate as measured by the WV School Climate Index. Differences al- 
so were found for eight of the 20 indicators the index comprises. As noted in the methods sec- 
tion, the annual changes in the school climate index among intervention schools are gauged rel- 
ative to ratings of nonintervention schools held constant at 2010-11 baseline levels. It will not be 
possible to determine if the improvements in the index scores observed in this study are genuine 
until data from nonintervention schools are collected again in the 2013-14 school year. 

Based on survey data, however, there appears to be a fairly wide gulf between students 
and staff in their perceptions of school climate improvements. Students were much more likely 
to report that conditions stayed about the same compared to the previous year, whereas staff 
were much more likely to report conditions had gotten better. 
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Discrepancies between student and staff perceptions of school conditions are not un- 
heard of. Waasdorp, Pas, O'Brennan, and Bradshaw (2011) found that although school staff were 
more likely than students to report having witnessed bullying in the school environment, they 
also were more likely than students to report feeling safe and a sense of belonging— both im- 
portant constructs of school climate. Despite being unable to compare student and staff 
measures of school climate directly, Mitchell, Bradshaw, and Leaf (2010) reported no associa- 
tion between student and staff ratings of overall school climate, and that teachers’ ratings of 
school climate may be more sensitive to classroom-level factors, such as disruptive behaviors, 
while students’ perceptions may be more sensitive to school-level factors, such as student mobil- 
ity, student-faculty ratio, and a change in principal. That staff and students were so far apart in 
their ratings of school climate improvement in the current study raises a question about what 
factors may be driving the perceptions of both groups. Moreover, once identified, can these fac- 
tors be targeted to further enhance schools’ efforts in school climate improvement? 

School Defined Behavior Expectations 

Establishing and reinforcing school-wide behavior norms or expectations is essential to 
the effective implementation of the WVMPSC. The norms and expectations provided by the S3 
intervention schools were, for the most part, clearly and positively stated. Most schools (62%) 
also had established five or fewer norms and expectations, which is in alignment with the guide- 
lines recommended under the School-wide Positive Behavior Intervention Supports framework 
(Algozzine, et al., 2010). It also appears that the intervention schools provided students and staff 
opportunities to contribute to establishing behavioral expectations and should be encouraged to 
continue to do so. Yet it is clear that in some schools, the established behavior expectations were 
too numerous, too wordy or unclear, unfriendly pronouncements of policy, or not mutually ex- 
clusive. 


Intervention schools also tended to rely more heavily on whole school approaches such 
as assemblies and morning announcements— or one-on-one re-teaching at the time of behavior 
incidents. While these indeed are worthwhile and should be continued, schools may benefit 
from expanding their approaches to communicate behavior expectations. 
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Recommendations 


1. Based on the list of behavior expectations provided by intervention schools, schools should 
direct attention to establishing or refining behavior expectations with the assistance of the 
school climate specialists (SCSs). Expectations should be five or fewer in number; 1-3 words 
in length; positively stated; comprehensive (apply to all students and staff, and all settings); 
and mutually exclusive. The findings from the fidelity assessment indicate that schools 
should place additional emphasis on efforts to build understanding of behavior expectation 
among school staff. 

2. Schools also should consider expanding their approaches for communicating and teaching 
behaviors that align with their expectations. By articulating specifically what appropriate 
behaviors look like relative to the established behavior expectations, developing specific les- 
son plans to guide the process of teaching appropriate behaviors, and modeling appropriate 
behaviors throughout the school environment, they will enhance the degree to which expec- 
tations are integrated into the day-to-day school experience. 

3. Essential to obtaining the best possible climate improvements is selecting and implementing 
interventions based on thorough assessments of factors contributing to problematic school 
climate issues. That SCSs assessed schools at lower levels of fidelity in this area than did 
school teams points to a need to redouble schools’ efforts to explore these contributing fac- 
tors and adjust their interventions accordingly. 

4. The gap between students’ and staffs’ perceptions of school climate improvement suggests 
an opportunity to cultivate a deeper understanding of the school environment from the per- 
spectives of these two groups. Identifying and leveraging factors driving the perceptions of 
both groups could provide direction for further efforts to improve school climate. 
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Appendix A. Strategic Steps of the WV Model for a Positive School 
Climate 


Table 11. Strategic Steps and Core Activities of the WV Model for a Positive School Climate Cross-Walk With 

Evidence Based Implementation Stages and Drivers 






Implementation 

Implementation Implementation 

Strategic step 

Core activity 

stage 

driver category 

driver(s) 

Organizing 

Establish an S3 team and train team members 

Exploration/ 

Competency 

Selection, 


on project design and components of the 

Positive School Climate Model. 

Installation 


Training 


Establish communication strategies among RESA, Installation 

Leadership 

Adaptive 


LEA, and school-based S3 team leadership. 





Establish ownership and commitment for the 

Installation 

Organization 

Facilitative 


project implementation process. 



Administration 

Assessment 

Establish data collection schedules and 

Exploration/ 

Organization 

Decision Support 


procedures. 

Installation 


Data System 


Conduct an environmental scan to describe 

Exploration 

Organization 

Decision Support 


events, trends, and relationships in the school's 
internal and external environment to assist in 
planning for the desired school climate. 



Data System 


Conduct a SWOT analysis to examine the 

Exploration 

Organization 

Decision Support 


school's internal strengths, weaknesses, 
environments, opportunities, and threats 
relative to the S3 project. 



Data System 


Use data collected from behavior intervention or Installation 

Organization 

Decision Support 


discipline referral forms, or other similar paper 
or electronic forms, for early detection, 
identification, and documentation of 
inappropriate behavior. 



Data System 


Conduct comprehensive needs assessment with 

Installation 

Organization 

Decision Support 


all relevant data (e.g., student surveys, student 
discipline data, DP-21, SWOT, etc.) to describe 
current conditions and define a desired school 

climate. 



Data System 


Prioritize project goals based on the district 
needs identified in the assessment process. 

Installation 

Leadership 

Technical 

Building support 

Inform school-level professional and support 

Installation 

Organization 

Systems 


staff about the S3 initiative and secure their 

commitment. 



Intervention 


Inform parents and community partners about 

Installation 

Organization 

Systems 


the S3 initiative and secure their commitment. 



Intervention 


Build understanding of S3 behavioral norms 
among school-level professional and support 
staff. 

Installation 

Competency 

Training 


Build understanding of S3 behavioral norms 
among parents and community partners. 

Installation 

Competency 

Training 


Define school-wide procedural expectations for 

Installation 

Organization 

Systems 


addressing student behavior in a consistent and 
deliberate manner. 



Intervention 

Table 11 continues on next page. 

Planning and 

Use comprehensive assessment results (e.g., the Installation/ 

Organization 

Decision Support 
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Table 11. Strategic Steps and Core Activities of the WV Model for a Positive School Climate Cross-Walk With 
Evidence Based Implementation Stages and Drivers 




Implementation 

Implementation Implementation 

Strategic step 

Core activity 

stage 

driver category 

driver(s) 

implementation 

environmental scan, student surveys, etc.) to 

Initial 


Data System 


identify factors contributing to school climate 
problems, set priorities, or plan S3 activities. 

Implementation 




Use contributing factors identified in the 

Initial 

Organization 

Decision Support 


assessment process to select appropriate 
interventions. 

Implementation 


Data System 


Establish clear plans to integrate selected 

Initial 

Organization 

Decision Support 


interventions school-wide. 

Implementation 


Data System 


Build capacity and train staff and parent/ 

Initial 

Competency 

Training 


community partners to implement interventions Implementation 




with fidelity school-wide and in the community. 





Implement project activities and selected 

Full 

Organization 

Systems 


interventions with fidelity school-wide. 

Implementation 


Intervention 


Collaborate with families and community organi- 

Full 

Organization 

Systems 


zations to create or enhance asset building op- 
portunities for students in school and communi- 
ty settings. 

Implementation 


Intervention 


Collaborate with community agencies to create 

Full 

Organization 

Systems 


or enhance intervention and treatment referral 
systems. 

Implementation 


Intervention 

Monitoring and 

Review or analyze data on an ongoing basis to 

Full 

Organization 

Decision Support 

evaluation 

make project adjustments. 

Implementation 


Data System 


Establish plans and processes for ensuring long- 

Full 

Organization 

Systems 


term project continuation and sustainability. 

Implementation 


Intervention 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV 
Model for a Positive School Climate 


Table 12. Fidelity Rating Scale for Core Activities Corresponding to the "Organizing" Strategic Step of the WV 

Model for a Positive School Climate 

Core activity 

Rating Fidelity level 

Description 

To what extent has your 
school established an S3 

1 Missing 

No attempt has yet been made to establish the school-level S3 
team or recruit members to serve. 

team and trained team 
members on project de- 

2 Weak Fidelity 

Establishment of an S3 team and recruitment of members has 
begun. No training has yet taken place. 

sign and components of 
the Positive School Cli- 
mate Model? 

3 Moderate Fidelity S3 team has been established and organized, with most members 
recruited. Some members have been trained on the project de- 
sign and components of the Positive School Climate Model. 


4 Strong Fidelity 

S3 team has been established and organized, and recruitment has 
been successfully completed. All members have been trained on 
the project design and the components of the Positive School 
Climate Model. 

To what extent has your 
school established com- 

1 Missing 

No communication strategies have yet been established among 
RESA, LEA, and school-based S3 team leadership. 

munication strategies 

among RESA, LEA, and 
school-based S3 team 
leadership? 

2 Weak Fidelity 

Communication strategies have been established between some 
of the partners (i.e., RESA, LEA, and school-based S3 team leader- 
ship). Communication strategies are not yet defined and/or are 
infrequently used. 


3 Moderate Fidelity Communication strategies have been established among RESA, 
LEA, and school-based S3 team leadership; are somewhat de- 
fined; and periodically used. 


4 Strong Fidelity 

Communication strategies have been established among RESA, 
LEA, and school-based S3 team leadership; they are clearly de- 
fined, and frequently used. 

To what extent has your 
school established own- 

1 Missing 

The school leadership has not yet taken ownership of the project 
or committed to its implementation. 

ership and commitment 
for the project implemen- 
tation process? 

2 Weak Fidelity 

The school leadership is in the early stages of the implementation 
process and shows some signs of ownership and commitment to 
the project. 


3 Moderate Fidelity The school has an S3 team that demonstrates a moderate level of 
ownership and commitment to the project implementation pro- 
cess. 


4 Strong Fidelity 

The school-level S3 team exhibits strong ownership and commit- 
ment to the project implementation process. 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 13. Fidelity Rating Scale for Core Activities Corresponding to the "Assessment" Strategic Step of the WV 
Model for a Positive School Climate 


Core activity 

Rating Fidelity level 

Description 

To what extent has your 

1 Missing 

No data collection schedules or procedures have been estab- 

school established data 


lished. 

collection schedules and 

2 Weak Fidelity 

A process to establish data collection schedules and procedures 

procedures? 


has been initiated, but is in its early stages. 


3 Moderate Fidelity 

Data collection schedules and procedures have been drafted 
and are in the process of refinement. 


4 Strong Fidelity 

Data collection schedules and procedures have been reviewed, 
further refined, and finalized. 

To what extent has your 

1 Missing 

Planning for and implementation of an environmental scan 

school conducted an Envi- 


have not begun. 

ronmental Scan to describe 

2 Weak Fidelity 

Planning for an environmental scan has been completed, but 

events, trends, and rela- 


no information has yet been collected. 

tionships in the school's 
internal and external envi- 

3 Moderate Fidelity 

An environmental scan is underway and is producing potential- 
ly useful information for the planning process. 

ronment to assist in plan- 
ning for the desired school 
climate? 

4 Strong Fidelity 

An environmental scan has been completed and all relevant 
information from the scan has been made available to the S3 
team to use in its planning. 

To what extent has your 

1 Missing 

Planning for and implementation of a SWOT analysis have not 

school conducted a 


begun. 

Strengths, Weaknesses, 

2 Weak Fidelity 

Planning for a SWOT analysis has been completed, but no in- 

Opportunities, and Threats 


formation has yet been collected. 

(SWOT) analysis to exam- 
ine the school's internal 

3 Moderate Fidelity 

A SWOT analysis is underway and is producing potentially use- 
ful information for the planning process. 

strengths, weaknesses, 

environments, opportuni- 
ties, and threats relative to 
the S3 project? 

4 Strong Fidelity 

A SWOT analysis has been completed and all relevant infor- 
mation from the scan has been made available to the S3 team 
to use in its planning. 

To what extent does the 

1 Missing 

Behavior intervention or discipline referral forms, or other simi- 

school use data collected 


lar paper or electronic forms, have not been introduced. 

from behavior intervention 

2 Weak Fidelity 

Behavior intervention or discipline referral forms, or other simi- 

or discipline referral forms, 


lar paper or electronic forms, have been introduced but they 

or other similar paper or 


are not yet being used. 

electronic forms, for early 
detection, identification, 
and documentation of in- 
appropriate behavior? 

3 Moderate Fidelity 

Behavior intervention or discipline referral forms, or other simi- 
lar paper or electronic forms, are being used for data collection 
by some school staff and data are beginning to be used for as- 
sessment or planning purposes. 


4 Strong Fidelity 

Behavior intervention or discipline referral forms, or other simi- 
lar paper or electronic forms, are widely used and data are fre- 
quently or always used for assessment or planning purposes. 

Table 13 continues on next page. 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 13. Fidelity Rating Scale for Core Activities Corresponding to the "Assessment" Strategic Step of the WV 
Model for a Positive School Climate 


Core activity 

Rating Fidelity level 

Description 

To what extent has your 
school conducted a com- 

1 Missing 

Planning for and implementation of a comprehensive needs 
assessment have not begun. 

prehensive needs assess- 
ment with all relevant data 

2 Weak Fidelity 

Planning for a comprehensive needs assessment has been 
completed, and relevant data are being gathered. 

(e.g., student surveys, stu- 
dent discipline data, DP-1, 

3 Moderate Fidelity 

Most relevant data have been gathered, and are being re- 
viewed for adequacy in describing current conditions. 

SWOT, etc.) to describe 
current conditions and 

define a desired school 

climate? 

4 Strong Fidelity 

All relevant data have been gathered and reviewed, and the 
baseline assessment of the data is complete. The product of 
these activities is a comprehensive description of current con- 
ditions and the definition of a desired school climate. 

To what extent has your 
school prioritized project 

1 Missing 

Prioritization of project goals based on district needs has not 
begun. 

goals based on the district 
needs identified in the 

2 Weak Fidelity 

Prioritization of project goals based on district needs is in early 
stages. 

assessment process? 

3 Moderate Fidelity 

Prioritization of project goals based on district needs is well 
underway. 


4 Strong Fidelity Prioritization of project goals based on district needs has been 
completed. 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 14. Fidelity Rating Scale for Core Activities Corresponding to the "Building Support" Strategic Step of the WV 
Model for a Positive School Climate 


Core activity Rating Fidelity level Description 


To what extent has your 
school informed school- 
level professional and 
support staff about the S3 
initiative and secured 

their commitment? 

1 Missing Planning for S3 initiative training for school-level professional and 

support staff has not begun. 

2 Weak Fidelity A plan is in place for training school-level professional and support 

staff on the S3 project, and is in the early stages of implementation. 

3 Moderate Some school-level professional and support staff have received train- 

Fidelity ing on the S3 project; some demonstrate commitment to the S3 pro- 

ject. 

4 Strong Fidelity All school-level professional and support staff have received training 

on the S3 project. Overall, there is a high level of commitment to the 
S3 project. 

To what extent has your 
school informed parents 
and community partners 
about the S3 initiative 

and secured their com- 
mitment? 

1 Missing Planning for activities to inform parents and community partners 

about the S3 initiative have not begun. 

2 Weak Fidelity A plan is in place for informing parents and community partners 

about the S3 initiative, and implementation has begun. 

3 Moderate Some parents and community partners have received information 

Fidelity about the S3 project; some demonstrate commitment to the S3 pro- 

ject. 

4 Strong Fidelity Many parents and community partners have received information 

about the S3 project; many demonstrate a high level of commitment 
to the S3 project. 

To what extent has your 
school built understand- 
ing of S3 behavioral 
norms among school-level 
professional and support 
staff? 

1 Missing Planning for S3 behavioral norm training for school-level professional 

and support staff has not begun. 

2 Weak Fidelity A plan is in place for building understanding about the S3 behavioral 

norms among school-level professional and support staff, and im- 
plementation is in early stages. Low levels of awareness exist among 
staff. 

3 Moderate Some efforts have been made to build awareness of S3 behavioral 

Fidelity norms among school-level professional and support staff. Moderate 

levels of awareness exist among staff. 

4 Strong Fidelity Numerous efforts have been made to build awareness of S3 behav- 

ioral norms among school-level professional and support staff. High 
levels of awareness exist among staff. 

To what extent has your 
school built understand- 
ing of S3 behavioral 
norms among parents 
and community partners? 

1 Missing Planning for activities to build understanding about S3 behavioral 

norms among parents and community partners has not begun. 

2 Weak Fidelity A plan is in place for building understanding about S3 behavioral 

norms among parents and community partners, and implementation 
has begun 

3 Moderate Some activities have taken place to build understanding of S3 behav- 

Fidelity ioral norms among parents and community partners. Moderate lev- 

els of awareness exist among this population. 

4 Strong Fidelity Numerous efforts have been made to build understanding of S3 be- 

havioral norms among parents and community partners. High levels 
of awareness exist among this population. 


Table 14 continues on next page. 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 14. Fidelity Rating Scale for Core Activities Corresponding to the "Building Support" Strategic Step of the WV 
Model for a Positive School Climate 


Core activity 

To what extent has your 
school defined school- 
wide procedural expecta- 
tions for addressing stu- 
dent behavior in a con- 
sistent and deliberate 
manner? 


Rating Fidelity level 

Description 

1 Missing 

Procedural expectations for addressing student behavior in a con- 
sistent and deliberate manner have not been defined. 

2 Weak Fidelity 

Work has begun on defining procedural expectations, but definitions 
are not yet sufficiently clear, complete, or always suitable. Few staff 
have adopted the procedural expectations. 

3 Moderate 
Fidelity 

Procedural expectations have been defined and have gained some 
clarity, but some misunderstanding and confusions about them con- 
tinues to exist. Some staff have adopted them. 

4 Strong Fidelity 

Procedural expectations for addressing student behavior in a con- 
sistent and deliberate manner have been clearly defined. A common 
understanding and interpretation of them is shared among all stake- 
holders. Procedural expectations have been adopted school-w 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 15. Fidelity Rating Scale for Core Activities Corresponding to the "Planning" Strategic Step of the WV Model 
for a Positive School Climate 


Core activity Rating Fidelity level Description 


To what extent has your 
school used comprehen- 
sive assessment results 
(e.g., the environmental 
scan, student surveys, 
etc.) to identify factors 
contributing to school 
climate problems, set 
priorities, or plan S3 activ- 
ities? 

1 Missing Assessment results are not used to identify factors contributing to 

school climate problems, set priorities, or plan S3 activities. 

2 Weak Fidelity Assessment results are used occasionally, but not as effectively as 

they could be to identify factors contributing to school climate prob- 
lems, set priorities, or plan S3 activities. 

3 Moderate Assessment results are used often, but not as effectively as they 

Fidelity could be to identify factors contributing to school climate problems, 

set priorities, or plan S3 activities. 

4 Strong Fidelity Assessment results are routinely used to identify factors contributing 

to school climate problems, and to logically set priorities, select in- 
terventions, or plan other S3 activities. 

To what extent has your 
school used contributing 
factors identified in the 
assessment process to 
select appropriate inter- 
ventions? 

1 Missing No interventions have been identified. 

2 Weak Fidelity Interventions are identified, but it is not clear how they address the 

factors contributing to the targeted issues. 

3 Moderate Interventions are identified with some alignment to identified con- 

Fidelity tributing factors or targeted problem, but there is evidence for selec- 

tion of other interventions. 

4 Strong Fidelity Interventions are identified that align well with the problem and con- 

tributing factors. 

To what extent has your 
school established clear 
plans to integrate select- 
ed interventions school- 
wide? 

1 Missing No plans have been established. 

2 Weak Fidelity Some plans have been established to integrate selected interven- 

tions school-wide, but they are unclear or not widely understood or 
followed. 

3 Moderate Clear plans have been established to integrate selected interventions 

Fidelity school-wide. Many staff and parent/community partners understand 

and follow the plans. 

4 Strong Fidelity Clear plans have been established to integrate selected interventions 

school-wide. Plans are closely followed by all staff and par- 
ent/community partners. 

To what extent has your 
school built capacity and 
trained staff and par- 
ent/community partners 
to implement interven- 
tions with fidelity school- 
wide and in the communi- 
ty? 

1 Missing No efforts have been undertaken to train or build the capacity of 

staff and parent/community partners to implement interventions. 

2 Weak Fidelity Some training or other effort has been undertaken to build staff and 

parent/community partner capacity to implement interventions, but 
they have been limited and few have participated. 

3 Moderate Most staff and parent/community partners have been trained and 

Fidelity have the capacity to implement identified interventions school-wide 

or in the community with fidelity. 

4 Strong Fidelity All staff and parent/community partners have been trained and have 

the capacity to implement identified interventions school-wide or in 
the community with fidelity. 


Table 15 continues on next page. 


To what extent has your 

1 Missing No implementation has taken place. 


36 I West Virginia Safe and Supportive Schools Project 






Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 15. Fidelity Rating Scale for Core Activities Corresponding to the "Planning" Strategic Step of the WV Model 
for a Positive School Climate 


Core activity 
school implemented pro- 
ject activities and selected 
interventions with fidelity 


Rating Fidelity level Description 

2 Weak Fidelity Project activities and selected interventions have been implemented, 
but not on a school-wide basis and/or there is evidence of low fideli- 
ty. 


school-wide? 

3 Moderate 
Fidelity 

Project activities and selected interventions have been implemented 
school-wide; some evidence exists that fidelity may be compromised. 


4 Strong Fidelity 

Project activities and selected interventions have been implemented 
school-wide with fidelity. 

To what extent has your 

1 Missing 

There is no collaboration with families and community organizations. 

school collaborated with 
families and community 
organizations to create or 

2 Weak Fidelity 

Collaboration with families and community organizations occurs but 
is weak or infrequent, or occurs in a limited number of school and 
community settings. 

enhance asset building 

3 Moderate 

Collaboration with families and community organizations occurs, but 

opportunities for students 
in school and community 
settings? 

Fidelity 

could be expanded or strengthened in some school and community 
settings. 

4 Strong Fidelity High levels of collaboration with families and community organiza- 
tions occurs in most or all school and community settings. 

To what extent has your 

1 Missing 

There is no collaboration with community agencies. 

school collaborated with 
community agencies to 

2 Weak Fidelity 

A limited level of collaboration occurs with community agencies to 
create or enhance intervention and treatment referral systems. 

create or enhance inter- 

3 Moderate 

A moderate level of collaboration occurs with community agencies to 

vention and treatment 
referral systems? 

Fidelity 

create or enhance intervention and treatment referral systems. 

4 Strong Fidelity A high level of collaboration occurs with community agencies to cre- 
ate or enhance intervention and treatment referral systems. 
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Appendix B. Fidelity Rating Scale for Core Activities of the WV Model for a Positive School Climate 


Table 16. Fidelity Rating Scale for Core Activities Corresponding to the "Monitoring and Evaluation" Strategic Step 
of the WV Model for a Positive School Climate 

Core activity Rating Fidelity level Description 


To what extent does your 1 Missing No review or analysis of data to make project adjustments has oc- 

school review or analyze curred. 


data on an ongoing basis 
to make project adjust- 
ments? 

2 Weak Fidelity 

Review or analysis of data to make project adjustment has been ini- 
tiated, but is in its early stages. No project adjustments have been 
made based on data. 


3 Moderate 
Fidelity 

Review or analysis of data has been initiated to make project ad- 
justments, but is done infrequently or could benefit from deeper 
exploration or reflection. 


4 Strong Fidelity 

Review or analysis of data is done on a frequent and ongoing basis to 
monitor progress. Decisions about project adjustments are based 
directly on the data review. 

To what extent has your 
school established plans 
and processes for ensur- 

1 Missing 

There has been no discussion about long-term project continuation 
or to develop a plan for sustaining the strategies and outcomes after 
funding has been depleted. 

ing long-term project con- 
tinuation and sustainabil- 
ity? 

2 Weak Fidelity 

Discussions about long term project continuation and sustainability 
have been initiated, but are limited in scope, and do not include spe- 
cific action steps for developing a sustainability plan. 


3 Moderate 
Fidelity 

Discussions about long term project continuation and sustainability 
have occurred, but include only some action steps for developing a 
sustainability plan, or steps may be infeasible or impractical to carry 
out. 


4 Strong Fidelity 

Discussions have produced a well-formulated plan about project 
continuation and sustainability that articulates specific, doable ac- 
tion steps for carrying out a sustainability plan. 
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Appendix C. School-Based S3 Team and School Climate Specialist 
Fidelity Assessment Ratings: 2011-2012 


Organizing for the S3 Initiative 


To what extent has the school established an S3 team and trained team members on project design and 
components of the Positive School Climate Model? 



Missing Weak Moderate Strong Too early Don't 

fidelity fidelity fidelity fidelity to tell know 


16 

14 

12 - 
_U) 

§ 10 - 
t 8 - 

o 

| 6 - 


2 - 



1 2 3 4 5 6 

Missing Weak Moderate Strong Too early Don't 

fidelity fidelity fidelity fidelity to tell know 


To what extent has the school established communication strategies among RESA, LEA, and school- 
based S3 team leadership? 




Missing Weak Moderate Strong Too early Don't 
fidelity J fidelity I fidelity fidelity tote!) know 


Missing Weak Moderate Strong nfoo early! Don't 

fidelity fidelity L fidelity fidelity to tell know 


To what extent has the school established ownership and commitment for the project implementation 
process? 


O 

O 


8 


16 
14 
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0 
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fidelity 
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fidelity 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


Needs Assessment Process 


To what extent has the school established data collection schedules and procedures? 


O 

o 

-C 

bi 

4 - 

o 




1 2 3 4 5 6 

Missing Weak Moderate Strong Too earlyl Don't 

fidelity fidelity fidelity fidelity to tell know 


To what extent has the school conducted an Environmental Scan to describe events, trends, and 
relationships in the school's internal and external environment to assist in planning for the desired 
school climate? 


V) 

O 

o 

-C 

K 

o 



1 2 3 4 5 6 

Missing Weak Moderate Strong Too early Don't 

fidelity I fidelity I I fidelity fidelity to tell know 



To what extent has the school conducted a Strengths, Weaknesses, Opportunities, and Threats (SWOT) 
analysis to examine the school's internal strengths, weaknesses, environments, opportunities, and 
threats relative to the S3 project? 



Missing 

Weak 

Moderate 

Strong 

Too early Don't 

Missing 

Weak 

Moderate 

Strong rjToo early Don't 

fidelity 

fidelity 

fidelity 

fidelity 

to tell know 

fidelity 

fidelity 

fidelity 

fidelity to tell know 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


To what extent does the school use data collected from behavior intervention or discipline referral 
forms, or other similar paper or electronic forms for early detection, identification, and documentation 
of inappropriate behaviors? 



Missing 

Weak 

Moderate 

Strong 

Too early 

Don't 

fidelity 

fidelity 

fidelity 

fidelity 

to tell 

know 


16 

14 - 
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.c 
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*5 8 H 
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Missing 
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HToo early 
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fidelity 

fidelity 

fidelity 

fidelity 

to tell 

know 


To what extent has the school conducted a comprehensive needs assessment with all relevant data 
(e.g., student surveys, student discipline data, DP-21, SWOT, BIFs, etc.) to describe current conditions 
and define a desired school climate? 


16 \ 
14 - 

12 I 


□ School climate specialists 
M School S3 teams 


2011 




T' — ~ 1 

1 

2 

1 1 

3 
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I 
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Missing 
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Weak 

fidelity 

Moderate 

fidelity 

Strong 

fidelity 

Too early 
to tell 

Don't 

know 


To what extent has the school prioritized project goals based on the district needs identified in the 
assessment process? 




1 2 3 4 5 6 

Missing n Weak Moderate Strong ["froo earlyl Don't 

fidelity fide lit y fidelity fidelity to tell know 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


Building Support for the S3 Initiative 


To what extent has the school informed school-level professional and support staff about the S3 
initiative and secured their commitment? 



123456 123456 
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To what extent has the school built understanding of S3 behavioral norms among school-level 
professional and support staff? 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


To what extent has the school built understanding of S3 behavioral norms among parents and 
community partners? 


V) 

O 

o 

-C 

K 

o 


□ School climate specialists 
■ School S3 teams 


2011 





14 - 
12 - 



1 2 3 4 5 6 
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fidelity | fidelity I fidelity fidelity to tell know 
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To what extent has the school defined school-wide procedural expectations for addressing student 
behavior in a consistent and deliberate manner? 



V) 

O 

O 


8 


□ School climate specialists 
■ School S3 teams 

1 
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fidelity 

fidelity 
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Systematic Planning 


To what extent has the school used comprehensive assessment results (e.g., the environmental scan, 
student surveys, etc.) to identify factors contributing to school climate problems, set priorities, or plan 
S3 activities? 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


To what extent has the school used contributing factors identified in the assessment process to select 
appropriate interventions? 


□ School climate specialists 
■ School SB teams 
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To what extent has the school established clear plans to integrate selected interventions school-wide? 




To what extent has the school built capacity and trained staff and parent/community partners to 
implement interventions with fidelity school-wide and in the community? 
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Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


To what extent has the school implemented project activities and selected interventions with fidelity 
school-wide? 


□ School climate specialists 
■ School S3 teams 


[2on] 




12 - 
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To what extent has the school collaborated with families and community organizations to create or 
enhance asset building opportunities for students in school and community settings? 
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To what extent has the school collaborated with community agencies to create or enhance intervention 
and treatment referral systems? 



2 

3 

4 

5 

6 

! Weak 

Moderate 

Strong 

nroo early 

Don't 

fidelity 

fidelity 

fidelity 

to tell 

know 



1 2 3 4 5 6 

Missing Weak Moderate Strong Too early Don't 

fidelity fidelity fidelity fidelity to tell know 


West Virginia Safe and Supportive Schools Project / 45 






Appendix C. School-Based S3 Team and School Climate Specialist Fidelity Assessment Ratings: 2011-2012 


Evaluation and Monitoring 


To what extent does the school review or analyze data on an ongoing basis to make project 
adjustments? 
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To what extent has the school established plans and processes for ensuring long-term project 
continuation and sustainability? 
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Appendix D. S3 Team and School Climate Specialist Fidelity 
Assessment Ratings, 2011 to 2012 


Table 17. Wilcoxon Signed Ranks Test of S3-Team Implementation Fidelity Rating Improvement Between 2011 and 2012 


Core activities 

Median Median 

2011 2012 

z 

P 

Establish an S3 team and train team members on project design and components of the 
Positive School Climate Model. 

4 

4 

-0.82 

0.414 

Establish communication strategies among RESA, LEA, and school-based S3 team leadership. 

3 

3 

-0.63 

0.527 

Establish ownership and commitment for the project implementation process. 

4 

4 

-1.63 

0.102 

Establish data collection schedules and procedures. 

3 

4 

-2.80 

0.005 

Conduct an environmental scan to describe events, trends, and relationships in the school's 
internal and external environment to assist in planning for the desired school climate. 

2 

3 

-1.39 

0.164 

Conduct a SWOT analysis to examine the school's internal strengths, weaknesses, 
environments, opportunities, and threats relative to the S3 project. 

3 

4 

-1.27 

0.205 

Use data collected from behavior intervention or discipline referral forms, or other similar 
paper or electronic forms, for early detection, identification, and documentation of 
inappropriate behavior? 

3 

4 

-2.98 

0.003 

Conduct comprehensive needs assessment with all relevant data to describe current 
conditions and define a desired school climate. 

2 

3 

-2.72 

0.006 

Prioritize project goals based on the district needs identified in the assessment process. 

2 

3 

-1.54 

0.125 

Inform school-level professional and support staff about the S3 initiative and secure their 
commitment. 

3 

4 

-2.74 

0.006 

Inform parents and community partners about the S3 initiative and secure their 
commitment. 

2 

3 

-2.81 

0.005 

Build understanding of S3 behavioral norms among school-level professional and support 
staff. 

3 

4 

-3.18 

0.001 

Build understanding of S3 behavioral norms among parents and community partners. 

2 

3 

-3.38 

0.001 

Define school-wide procedural expectations for addressing student behavior in a consistent 
and deliberate manner. 

3 

4 

-2.94 

0.003 

Use comprehensive assessment results (e.g., the environmental scan, student surveys, etc.) 
to identify factors contributing to school climate problems, set priorities, or plan S3 
activities. 

1 

4 

-3.10 

0.002 

Use contributing factors identified in the assessment process to select appropriate 
interventions. 

3 

3 

-2.54 

0.011 

Establish clear plans to integrate selected interventions school-wide. 

2 

3 

-2.17 

0.030 

Build capacity and train staff and parent/community partners to implement interventions 
with fidelity school-wide and in the community. 

2 

2 

-2.54 

0.011 

Implement project activities and selected interventions with fidelity school-wide. 

2 

4 

-2.33 

0.020 

Collaborate with families and community organizations to create or enhance asset building 
opportunities for students in school and community settings. 

1 

3 

-2.77 

0.006 

Collaborate with community agencies to create or enhance intervention and treatment 
referral systems. 

2 

3 

-2.22 

0.027 

Review or analyze data on an ongoing basis to make project adjustments. 

2 

4 

-2.87 

0.004 

Establish plans and processes for ensuring long-term project continuation and sustainability. 

2 

3 

-0.14 

0.891 

Fidelity Rating on a 4-point scale: 1 - Missing; 2 = Weak Fidelity; 3 = Moderate Fidelity; 4 = Strong Fidelity 
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Appendix D. S3 Team and School Climate Specialist Fidelity Assessment Ratings , 2011 to 2012 


Table 18. Wilcoxon Signed Ranks Test of School Climate Specialist Implementation Fidelity Rating Improvement 
Between 2011 and 2012 


Core activities 

Median 

2011 

Median 

2012 

z 

P 

Establish an S3 team and train team members on project design and components of the 
Positive School Climate Model. 

4 

4 

-0.82 

0.414 

Establish communication strategies among RESA, LEA, and school-based S3 team 
leadership. 

3 

3 

-0.43 

0.666 

Establish ownership and commitment for the project implementation process. 

4 

4 

-1.38 

0.167 

Establish data collection schedules and procedures. 

2 

3 

-3.13 

0.002 

Conduct an environmental scan to describe events, trends, and relationships in the 
school's internal and external environment to assist in planning for the desired school 
climate. 

3 

3 

-2.23 

0.026 

Conduct a SWOT analysis to examine the school's internal strengths, weaknesses, 
environments, opportunities, and threats relative to the S3 project. 

4 

4 

-2.01 

0.044 

Use data collected from behavior intervention or discipline referral forms, or other 
similar paper or electronic forms, for early detection, identification, and documentation 
of inappropriate behavior? 

2 

3 

-2.99 

0.003 

Conduct comprehensive needs assessment with all relevant data to describe current 
conditions and define a desired school climate. 

2 

3 

-3.27 

0.001 

Prioritize project goals based on the district needs identified in the assessment process. 

2 

3 

-2.76 

0.006 

Inform school-level professional and support staff about the S3 initiative and secure 
their commitment. 

2 

3 

-2.96 

0.003 

Inform parents and community partners about the S3 initiative and secure their 
commitment. 

2 

3 

-3.39 

0.001 

Build understanding of S3 behavioral norms among school-level professional and 
support staff. 

2 

3 

-3.24 

0.001 

Build understanding of S3 behavioral norms among parents and community partners. 

1 

3 

-3.22 

0.001 

Define school-wide procedural expectations for addressing student behavior in a 
consistent and deliberate manner. 

2 

3 

-3.36 

0.001 

Use comprehensive assessment results (e.g., the environmental scan, student surveys, 
etc.) to identify factors contributing to school climate problems, set priorities, or plan S3 
activities. 

2 

3 

-2.82 

0.005 

Use contributing factors identified in the assessment process to select appropriate 
interventions. 

2 

3 

-1.98 

0.048 

Establish clear plans to integrate selected interventions school-wide. 

2 

3 

-2.60 

0.009 

Build capacity and train staff and parent/community partners to implement 
interventions with fidelity school-wide and in the community. 

1 

3 

-2.91 

0.004 

Implement project activities and selected interventions with fidelity school-wide. 

1 

3 

-2.84 

0.004 

Collaborate with families and community organizations to create or enhance asset 
building opportunities for students in school and community settings. 

2 

3 

-3.07 

0.002 

Collaborate with community agencies to create or enhance intervention and treatment 
referral systems. 

2 

3 

-3.31 

0.001 

Review or analyze data on an ongoing basis to make project adjustments. 

2 

3 

-3.37 

0.001 

Establish plans and processes for ensuring long-term project continuation and 
sustainability. 

1 

2 

-3.21 

0.001 

Fidelity Rating on a 4-point scale: 1 - Missing; 2 = Weak Fidelity; 3 = Moderate Fidelity; 4 

= Strong Fidelity 
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Appendix E. Comparison of S3 Team and School Climate Specialist 
Fidelity Assessments Ratings, 2011 and 2012 


Table 19. Mann-Whitney U Test for S3 Team and School Climate Specialist Implementation Fidelity Rating 
Differences in 2011 


Core activity 

U 

z 

P 

Establish an S3 team and train team members on project design and components of the 
Positive School Climate Model. 

106.0 

-0.42 

0.678 

Establish communication strategies among RESA, LEA, and school-based S3 team leadership. 

105.5 

-0.31 

0.758 

Establish ownership and commitment for the project implementation process. 

81.0 

-1.57 

0.117 

Establish data collection schedules and procedures. 

72.5 

-1.76 

0.078 

Conduct an environmental scan to describe events, trends, and relationships in the school's 
internal and external environment to assist in planning for the desired school climate. 

80.5 

-1.41 

0.159 

Conduct a SWOT analysis to examine the school's internal strengths, weaknesses, 
environments, opportunities, and threats relative to the S3 project. 

110.5 

-0.09 

0.929 

Use data collected from behavior intervention or discipline referral forms, or other similar 
paper or electronic forms, for early detection, identification, and documentation of 
inappropriate behavior? 

82.5 

-1.31 

0.189 

Conduct comprehensive needs assessment with all relevant data to describe current 
conditions and define a desired school climate. 

106.5 

-0.27 

0.790 

Prioritize project goals based on the district needs identified in the assessment process. 

109.0 

-0.15 

0.878 

Inform school-level professional and support staff about the S3 initiative and secure their 
commitment. 

68.0 

-1.95 

0.051 

Inform parents and community partners about the S3 initiative and secure their 
commitment. 

60.5 

-2.31 

0.021 

Build understanding of S3 behavioral norms among school-level professional and support 
staff. 

46.5 

-2.90 

0.004 

Build understanding of S3 behavioral norms among parents and community partners. 

94.0 

-0.84 

0.402 

Define school-wide procedural expectations for addressing student behavior in a consistent 
and deliberate manner. 

80.0 

-1.48 

0.139 

Use comprehensive assessment results (e.g., the environmental scan, student surveys, etc.) 
to identify factors contributing to school climate problems, set priorities, or plan S3 
activities. 

90.5 

-0.94 

0.347 

Use contributing factors identified in the assessment process to select appropriate 
interventions. 

98.0 

-0.62 

0.532 

Establish clear plans to integrate selected interventions school-wide. 

81.0 

-1.41 

0.158 

Build capacity and train staff and parent/community partners to implement interventions 
with fidelity school-wide and in the community. 

105.0 

-0.33 

0.741 

Implement project activities and selected interventions with fidelity school-wide. 

108.0 

-0.19 

0.847 

Collaborate with families and community organizations to create or enhance asset building 
opportunities for students in school and community settings. 

78.5 

-1.48 

0.139 

Collaborate with community agencies to create or enhance intervention and treatment 
referral systems. 

103.0 

-0.41 

0.678 

Review or analyze data on an ongoing basis to make project adjustments. 

74.0 

-1.70 

0.088 

Establish plans and processes for ensuring long-term project continuation and sustainability. 

38.5 

-3.28 

0.001 
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Appendix E. Comparison of S3 Team and School Climate Specialist Fidelity Assessments Ratings , 2011 and 2012 


Table 20. Mann-Whitney U Test for S3 Team and School Climate Specialist Implementation Fidelity Rating 
Differences in 2012 


Core activity 

U 

z 

P 

Establish an S3 team and train team members on project design and components of the 
Positive School Climate Model. 

105.0 

-0.60 

0.550 

Establish communication strategies among RESA, LEA, and school-based S3 team 
leadership. 

99.5 

-0.65 

0.517 

Establish ownership and commitment for the project implementation process. 

75.0 

-2.41 

0.016 

Establish data collection schedules and procedures. 

94.5 

-0.85 

0.396 

Conduct an environmental scan to describe events, trends, and relationships in the school's 
internal and external environment to assist in planning for the desired school climate. 

88.5 

-1.05 

0.293 

Conduct a SWOT analysis to examine the school's internal strengths, weaknesses, 
environments, opportunities, and threats relative to the S3 project. 

92.0 

-1.05 

0.292 

Use data collected from behavior intervention or discipline referral forms, or other similar 
paper or electronic forms, for early detection, identification, and documentation of 
inappropriate behavior? 

75.0 

-1.81 

0.070 

Conduct comprehensive needs assessment with all relevant data to describe current 
conditions and define a desired school climate. 

110.5 

-0.11 

0.915 

Prioritize project goals based on the district needs identified in the assessment process. 

101.5 

-0.53 

0.596 

Inform school-level professional and support staff about the S3 initiative and secure their 
commitment. 

90.0 

-1.09 

0.277 

Inform parents and community partners about the S3 initiative and secure their 
commitment. 

70.5 

-2.26 

0.024 

Build understanding of S3 behavioral norms among school-level professional and support 
staff. 

58.5 

-2.58 

0.010 

Build understanding of S3 behavioral norms among parents and community partners. 

91.0 

-1.71 

0.087 

Define school-wide procedural expectations for addressing student behavior in a consistent 
and deliberate manner. 

67.5 

-2.20 

0.028 

Use comprehensive assessment results (e.g., the environmental scan, student surveys, etc.) 
to identify factors contributing to school climate problems, set priorities, or plan S3 
activities. 

57.5 

-2.50 

0.012 

Use contributing factors identified in the assessment process to select appropriate 
interventions. 

49.5 

-3.14 

0.002 

Establish clear plans to integrate selected interventions school-wide. 

73.0 

-1.83 

0.067 

Build capacity and train staff and parent/community partners to implement interventions 
with fidelity school-wide and in the community. 

109.5 

-0.14 

0.891 

Implement project activities and selected interventions with fidelity school-wide. 

71.5 

-1.86 

0.063 

Collaborate with families and community organizations to create or enhance asset building 
opportunities for students in school and community settings. 

109.0 

-0.18 

0.857 

Collaborate with community agencies to create or enhance intervention and treatment 
referral systems. 

104.5 

-0.38 

0.704 

Review or analyze data on an ongoing basis to make project adjustments. 

59.5 

-2.61 

0.009 

Establish plans and processes for ensuring long-term project continuation and 
sustainability. 

71.0 

-1.94 

0.053 
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Appendix F. Chi Square Analysis of Student and Staff Perceptions 
of School Climate Change 


Table 21. Student Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School 
Years 

Percent 
Percent expected 


Indicator 

Response 

students 

value 

2 

X 

df 

P 

Relationships among students 

A Little or A Lot Better 

45.3 a 

33.3 

101.8 

2 

0.00 


Stayed About the Same 

43.7 a 

33.3 





A Little or A Lot Worse 

11.1 a 

33.3 




Relationships among students and 

A Little or A Lot Better 

37.6 

33.3 

96.8 

2 

0.00 

staff 

Stayed About the Same 

50.4 a 

33.3 





A Little or A Lot Worse 

12.1 a 

33.3 




Respect for racial, ethnic, or cultural 

A Little or A Lot Better 

32.4 

33.3 

158.3 

2 

0.00 

diversity 

Stayed About the Same 

59.4 a 

33.3 





A Little or A Lot Worse 

8.2 a 

33.3 




Meaningful opportunities for 

A Little or A Lot Better 

40.2 a 

33.3 

132.9 

2 

0.00 

participation in school 

Stayed About the Same 

51.2 a 

33.3 





A Little or A Lot Worse 

8.7 a 

33.3 




Bullying at school 

A Little or A Lot Better 

35.8 

33.3 

62.4 

2 

0.00 


Stayed About the Same 

48.1 a 

33.3 





A Little or A Lot Worse 

16.1 a 

33.3 




Students' perceptions of personal 

A Little or A Lot Better 

32.9 

33.3 

159.4 

2 

0.00 

safety at school 

Stayed About the Same 

59.1 a 

33.3 





A Little or A Lot Worse 

8.0 a 

33.3 




Students' avoiding school activities for 

A Little or A Lot Better 

33.2 

33.3 

169.2 

2 

0.00 

fear of being harmed 

Stayed About the Same 

59.5 a 

33.3 





A Little or A Lot Worse 

7.3 a 

33.3 




Students' avoiding specific places in 

A Little or A Lot Better 

32.5 

33.3 

176.7 

2 

0.00 

school for fear of being harmed 

Stayed About the Same 

60.6 a 

33.3 





A Little or A Lot Worse 

7.0 a 

33.3 




Students' being called hate-related 

A Little or A Lot Better 

30.9 

33.3 

122.2 

2 

0.00 

words and seeing hate-related graffiti 

Stayed About the Same 

57.7 a 

33.3 





A Little or A Lot Worse 

11.4 a 

33.3 




Physical fights on school property 

A Little or A Lot Better 

41.9 a 

33.3 

81.5 

2 

0.00 


Stayed About the Same 

45.1 a 

33.3 





A Little or A Lot Worse 

13.1 a 

33.3 




Students' carrying weapons on school 

A Little or A Lot Better 

36.1 

33.3 

168.8 

2 

0.00 

property 

Stayed About the Same 

57.2 a 

33.3 





A Little or A Lot Worse 

6.7 a 

33.3 




Safety and security measures 

A Little or A Lot Better 

38.8 a 

33.3 

160.1 

2 

0.00 

observed at school 

Stayed About the Same 

54.4 a 

33.3 





A Little or A Lot Worse 

6.9 a 

33.3 





Table 21 continues on next page. 
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Appendix F. Chi Square Analysis of Student and Staff Perceptions of School Climate Change 


Table 21. Student Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School 
Years 

Percent 
Percent expected 


Indicator 

Response 

students 

value 

2 

X 

df 

P 

Teachers threatened with injury or 

A Little or A Lot Better 

33.6 

33.3 

206.6 

2 

0.00 

physical attack by students 

Stayed About the Same 

61.5 a 

33.3 





A Little or A Lot Worse 

4.8 3 

33.3 




Threats and injuries with weapons on 

A Little or A Lot Better 

31.9 

33.3 

169.7 

2 

0.00 

school property 

Stayed About the Same 

60.6 3 

33.3 





A Little or A Lot Worse 

7.5 3 

33.3 




Violent and other crime incidents at 

A Little or A Lot Better 

32.3 

33.3 

187.2 

2 

0.00 

school 

Stayed About the Same 

61.5 a 

33.3 





A Little or A Lot Worse 

6.3 a 

33.3 




Gang activity at school 

A Little or A Lot Better 

31.0 

33.3 

189.0 

2 

0.00 


Stayed About the Same 

62.5 a 

33.3 





A Little or A Lot Worse 

6.4 a 

33.3 




Students' use of tobacco/alcohol/ 

A Little or A Lot Better 

26.1 3 

33.3 

30.3 

2 

0.00 

drugs on school property 

Stayed About the Same 

47.7 3 

33.3 





A Little or A Lot Worse 

26.2 3 

33.3 




School conditions (schoolyard and 

A Little or A Lot Better 

31.8 

33.3 

67.9 

2 

0.00 

buildings clean and in good condition) 

Stayed About the Same 

51.6 a 

33.3 





A Little or A Lot Worse 

16.6 a 

33.3 




The school being a supportive 

A Little or A Lot Better 

36.3 

33.3 

148.6 

2 

0.00 

academic environment 

Stayed About the Same 

55.7 a 

33.3 





A Little or A Lot Worse 

8.0 a 

33.3 




Students' physical or mental health 

A Little or A Lot Better 

30.2 

33.3 

134.2 

2 

0.00 


Stayed About the Same 

59.2 a 

33.3 





A Little or A Lot Worse 

10.6 a 

33.3 




Discipline problems reported at school A Little or A Lot Better 

31.9 

33.3 

67.9 

2 

0.00 

(disruptive behavior or cutting 

Stayed About the Same 

51.5 a 

33.3 




classes/truancy) 

A Little or A Lot Worse 

16.5 a 

33.3 




Disciplinary actions taken by schools 

A Little or A Lot Better 

35.2 

33.3 

89.9 

2 

0.00 

(suspensions, detention, etc.) 

Stayed About the Same 

51.7 3 

33.3 





A Little or A Lot Worse 

13.1 3 

33.3 





a Differs significantly from the expected value at p < 0.05. 
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Appendix F. Chi Square Analysis of Student and Staff Perceptions of School Climate Change 


Table 22. Staff Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School Years 

Indicator 

Response 

Percent 

staff 

Percent 

expected 

value 

2 

X 

df 

P 

Relationships among students 

A Little or A Lot Better 

58.3 a 

33.3 

234.3 

2 

0.00 


Stayed About the Same 

39.1 a 

33.3 





A Little or A Lot Worse 

2.6 a 

33.4 




Relationships among students and staff 

A Little or A Lot Better 

60.9 a 

33.3 

200.5 

2 

0.00 


Stayed About the Same 

33.9 

33.3 





A Little or A Lot Worse 

5.2 a 

33.4 




Respect for racial, ethnic, or cultural 

A Little or A Lot Better 

37.9 

33.3 

256.4 

2 

0.00 

diversity 

Stayed About the Same 

60.5 a 

33.3 





A Little or A Lot Worse 

1.6 a 

33.4 




Meaningful opportunities for 

A Little or A Lot Better 

64.7 a 

33.3 

274.3 

2 

0.00 

participation in school 

Stayed About the Same 

33.8 

33.3 





A Little or A Lot Worse 

1.4 a 

33.4 




Bullying at school 

A Little or A Lot Better 

48.3 a 

33.3 

152.5 

2 

0.00 


Stayed About the Same 

44.8 a 

33.3 





A Little or A Lot Worse 

ro 

00 

33.4 




Students' perceptions of personal 

A Little or A Lot Better 

56.5 a 

33.3 

238.9 

2 

0.00 

safety at school 

Stayed About the Same 

41.3 a 

33.3 





A Little or A Lot Worse 

2.2 a 

33.4 




Students' avoiding school activities for 

A Little or A Lot Better 

50.4 a 

33.3 

248.9 

2 

0.00 

fear of being harmed 

Stayed About the Same 

48.3 a 

33.3 





A Little or A Lot Worse 

1.3 a 

33.4 




Students' avoiding specific places in 

A Little or A Lot Better 

49.3 a 

33.3 

239.4 

2 

0.00 

school for fear of being harmed 

Stayed About the Same 

49.0 a 

33.3 





A Little or A Lot Worse 

1.7 a 

33.4 




Students' being called hate-related 

A Little or A Lot Better 

50.6 a 

33.3 

197.7 

2 

0.00 

words and seeing hate-related graffiti 

Stayed About the Same 

45.5 a 

33.3 





A Little or A Lot Worse 

3.9 a 

33.4 




Physical fights on school property 

A Little or A Lot Better 

63.4 a 

33.3 

227.1 

2 

0.00 


Stayed About the Same 

32.7 

33.3 





A Little or A Lot Worse 

3.9 a 

33.4 




Students' carrying weapons on school 

A Little or A Lot Better 

47.7 a 

33.3 

253.5 

2 

0.00 

property 

Stayed About the Same 

51.3 a 

33.3 





A Little or A Lot Worse 

1.0 a 

33.4 




Safety and security measures observed 

A Little or A Lot Better 

61.0 a 

33.3 

253.2 

2 

0.00 

at school 

Stayed About the Same 

37.1 

33.3 





A Little or A Lot Worse 

1.9 a 

33.4 




Teachers threatened with injury or 

A Little or A Lot Better 

46.1 a 

33.3 

203.5 

2 

0.00 

physical attack by students 

Stayed About the Same 

50.4 a 

33.3 





A Little or A Lot Worse 

3.5 a 

33.4 




Threats and injuries with weapons on 

A Little or A Lot Better 

46.0 a 

33.3 

256.6 

2 

0.00 

school property 

Stayed About the Same 

53.1 a 

33.3 





A Little or A Lot Worse 

0.9 a 

33.4 







Table 22 continues on next page. 

Gang activity at school 

A Little or A Lot Better 

41.2 a 

33.3 

267.8 

2 

0.00 
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Appendix F. Chi Square Analysis of Student and Staff Perceptions of School Climate Change 


Table 22. Staff Perceptions of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 School Years 

Indicator 

Response 

Percent 

staff 

Percent 

expected 

value 

2 

z 

df 

P 


Stayed About the Same 

58.3 a 

33.3 





A Little or A Lot Worse 

0.6 a 

33.4 




Students' use of tobacco/alcohol/drugs A Little or A Lot Better 

33.4 

33.3 

61.3 

2 

0.00 

on school property 

Stayed About the Same 

49.9 a 

33.3 





A Little or A Lot Worse 

16.8 a 

33.4 




School conditions (schoolyard and 

A Little or A Lot Better 

46.2 a 

33.3 

101.5 

2 

0.00 

buildings clean and in good condition) 

Stayed About the Same 

42.8 a 

33.3 





A Little or A Lot Worse 

11.0 a 

33.4 




The school being a supportive academic A Little or A Lot Better 

62.7 a 

33.3 

225.6 

2 

0.00 

environment 

Stayed About the Same 

33.5 

33.3 





A Little or A Lot Worse 

3.8 a 

33.4 




Students' physical or mental health 

A Little or A Lot Better 

42.4 a 

33.3 

191.4 

2 

0.00 


Stayed About the Same 

53.1 a 

33.3 





A Little or A Lot Worse 

4.5 a 

33.4 




Discipline problems reported at school 

A Little or A Lot Better 

49.8 a 

33.3 

79.5 

2 

0.00 

(disruptive behavior or cutting 

Stayed About the Same 

36.3 

33.3 




classes/truancy) 

A Little or A Lot Worse 

13.9 a 

33.4 




Disciplinary actions taken by schools 

A Little or A Lot Better 

55.7 a 

33.3 

118.9 

2 

0.00 

(suspensions, detention, etc.) 

Stayed About the Same 

33.4 

33.3 





A Little or A Lot Worse 

10.9 a 

33.4 




a Differs significantly from the expected value at p < 0.05. 
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Appendix F. Chi Square Analysis of Student and Staff Perceptions of School Climate Change 


Table 23. Student and Staff Comparisons of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 
School Years 




Percent 

Percent 

Percent 




Indicator 

Response 

students 

staff 

total 

2 

X 

df 

P 

Relationships among students 

A Little or A Lot Better 

45.3 a 

58.3 b 

51.8 

49.669 

2 

0.00 


Stayed About the Same 

43.7 

39.1 

41.4 





A Little or A Lot Worse 

11.1 a 

2.6 b 

6.8 




Relationships among students 

A Little or A Lot Better 

37.6 a 

60.9 b 

49.2 

80.164 

2 

0.00 

and staff 

Stayed About the Same 

50.4 a 

33.9 b 

42.1 





A Little or A Lot Worse 

12.1 a 

5.2 b 

8.6 




Respect for racial, ethnic, or 

A Little or A Lot Better 

32.4 a 

37.9 b 

35.1 

33.977 

2 

0.00 

cultural diversity 

Stayed About the Same 

59.4 

60.5 

60.0 





A Little or A Lot Worse 

8.2 a 

1.6 b 

4.9 




Meaningful opportunities for 

A Little or A Lot Better 

40.2 3 

64.7 b 

52.5 

100.388 

2 

0.00 

participation in school 

Stayed About the Same 

51.2 3 

33.8 b 

42.5 





A Little or A Lot Worse 

8.7 3 

1.4 b 

5.0 




Bullying at school 

A Little or A Lot Better 

35.8 3 

48.3 b 

42.1 

39.5 

2 

0.00 


Stayed About the Same 

48.1 

44.8 

46.5 





A Little or A Lot Worse 

16.1 3 

6.8 b 

11.5 




Students' perceptions of personal A Little or A Lot Better 

32.9 a 

56.5 b 

44.8 

88.027 

2 

0.00 

safety at school 

Stayed About the Same 

59.1 a 

41.3 b 

50.1 





A Little or A Lot Worse 

8.0 a 

2.2 b 

5.1 




Students' avoiding school 

A Little or A Lot Better 

33.2 a 

50.4 b 

41.8 

60.795 

2 

0.00 

activities for fear of being 

Stayed About the Same 

59.5 a 

48.3 b 

53.9 




harmed 

A Little or A Lot Worse 

7.3 a 

1.3 b 

4.3 




Students' avoiding specific places 

A Little or A Lot Better 

32.5 3 

49.3 b 

40.9 

53.924 

2 

0.00 

in school for fear of being 

Stayed About the Same 

60.6 3 

49.0 b 

54.8 




harmed 

A Little or A Lot Worse 

7.0 3 

1.7 b 

4.3 




Students' being called hate- 

A Little or A Lot Better 

30.9 3 

50.6 b 

40.7 

67.34 

2 

0.00 

related words and seeing hate- 

Stayed About the Same 

57.7 3 

45.5 b 

51.6 




related graffiti 

A Little or A Lot Worse 

11.4 3 

3.9 b 

7.7 




Physical fights on school property A Little or A Lot Better 

41.9 3 

63.4 b 

52.7 

78.439 

2 

0.00 


Stayed About the Same 

45.1 3 

32.7 b 

38.8 





A Little or A Lot Worse 

13.1 3 

3.9 b 

8.5 




Students' carrying weapons on 

A Little or A Lot Better 

36.1 3 

47.7 b 

41.9 

41.808 

2 

0.00 

school property 

Stayed About the Same 

57.2 3 

51.3 b 

54.3 





A Little or A Lot Worse 

6.7 3 

1.0 b 

3.9 




Safety and security measures 

A Little or A Lot Better 

38.8 3 

61.0 b 

49.9 

75.973 

2 

0.00 

observed at school 

Stayed About the Same 

54.4 3 

37.1 b 

45.7 





A Little or A Lot Worse 

6.9 3 

1.9 b 

4.4 




Teachers threatened with injury 

A Little or A Lot Better 

33.6 a 

46.1 b 

39.9 

22.349 

2 

0.00 

or physical attack by students 

Stayed About the Same 

61.5 a 

50.4 b 

56.0 





A Little or A Lot Worse 

4.8 

3.5 

4.2 




Threats and injuries with 

A Little or A Lot Better 

31.9 3 

46.0 b 

38.9 

56.197 

2 

0.00 

weapons on school property 

Stayed About the Same 

60.6 3 

53.1 b 

56.9 





A Little or A Lot Worse 

7.5 3 

0.9 b 

4.2 




Table 23 continues on next page. 

Violent and other crime incidents 

A Little or A Lot Better 

32.3 3 

49.9 b 

41.0 

64.474 

2 

0.00 


West Virginia Safe and Supportive Schools Project / 55 


Appendix F. Chi Square Analysis of Student and Staff Perceptions of School Climate Change 


Table 23. Student and Staff Comparisons of School Climate Conditions in S3 Intervention Schools, 2011 and 2012 
School Years 


Indicator 

Response 

Percent 

students 

Percent 

staff 

Percent 

total 

2 

X 

df 

P 

at school 

Stayed About the Same 

61.5 a 

49.4 b 

55.5 





A Little or A Lot Worse 

6.3 3 

0.7 b 

3.5 




Gang activity at school 

A Little or A Lot Better 

31.0 a 

41.2 b 

36.1 

43.704 

2 

0.00 


Stayed About the Same 

62.5 

58.3 

60.4 





A Little or A Lot Worse 

6.4 a 

0.6 

3.5 




Students' use of tobacco/alcohol/ A Little or A Lot Better 

26.1 3 

33.4 

29.8 

20.889 

2 

0.00 

drugs on school property 

Stayed About the Same 

47.7 3 

49.9 

48.8 





A Little or A Lot Worse 

26.2 3 

16.8 b 

21.5 




School conditions (schoolyard 

A Little or A Lot Better 

31.8 3 

46.2 b 

39.1 

31.862 

2 

0.00 

and buildings clean and in good 

Stayed About the Same 

51.6 3 

42.8 b 

47.2 




condition) 

A Little or A Lot Worse 

16.6 3 

11.0 b 

13.8 




The school being a supportive 

A Little or A Lot Better 

36.3 a 

62.7 b 

49.5 

96.749 

2 

0.00 

academic environment 

Stayed About the Same 

55.7 a 

33.5 b 

44.6 





A Little or A Lot Worse 

8.0 a 

3.8 b 

5.9 




Students' physical or mental 

A Little or A Lot Better 

30.2 a 

42.4 b 

36.3 

33.247 

2 

0.00 

health 

Stayed About the Same 

59.2 a 

53.1 b 

56.1 





A Little or A Lot Worse 

10.6 a 

4.5 b 

7.6 




Discipline problems reported at 

A Little or A Lot Better 

31.9 a 

49.8 b 

40.9 

46.651 

2 

0.00 

school (disruptive behavior or 

Stayed About the Same 

51.5 a 

36.3 b 

43.9 




cutting classes/truancy) 

A Little or A Lot Worse 

16.5 

13.9 

15.2 




Disciplinary actions taken by 

A Little or A Lot Better 

35.2 a 

55.7 b 

45.5 

60.325 

2 

0.00 

schools (suspensions, detention, 

Stayed About the Same 

51.7 a 

33.4 b 

42.5 




etc.) 

A Little or A Lot Worse 

13.1 

10.9 

12.0 





Each superscript letter denotes a subset of role categories whose column proportions do not differ significantly from 
each other at p < 0.05. Column cells with different superscript letters differ at p < 0.05. 
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